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(54) Image processing device and method for identifying an input image, and copier scanner and printer 
including same. 



previously specified number of positioning 
marks and dummy marks which identify the 
location of the pattern are provided in the 
second embodiment 

It will be able to detect a wide variety of 
documents and perform the required proces- 
sing at high speed. 
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(57) The first object of this invention is to provide 
an image processing device which can reliably 
detect documents which may not be copied, 
which has a very small probability of false posi- 
tives, and which can handle newly issued docu- 
ments which may not be copied. 

^1 The second object of this invention is to 

^ provide an image processing scheme which 
would accurately obtain the location of a pat- 

^ tern indicating a document to be detected ; 

i& which would extract that pattern : which would 
. determine correctly whether it is the right pat- 

2 ter " ; and which make it difficult for anyone to 

<0 tamper with the pattern. 

0 To achieve such objects, a specified pattern is 

imprinted before use on a document which may 
^ not be copied in the first embodiment, and a 
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Field Of The Invention 

This invention concerns an image processing de- 
vice and Method for identifying an input image, and 
Copier Scanner and Printer including same. More pre- 
cisely, this invention concerns an image processing 
device and method for identifying an input image 
ideally suited for scanning documents which may not 
be copied, such as bank notes or negotiable securi- 
ties, and preventing a copier from printing out copies 
of them. It also concerns a copier, scanner and printer 
in which such a device is installed. 

This invention involves two major embodiments. 
The first embodiment concerns an image processing 
device equipped with a device to detect a pattern 
within image data it has received which is a potential 
match for a specified pattern; a device to extract this 
pattern, once detected, from the aforesaid image 
data; and a matching device which performs a fuzzy 
inference using fuzzy knowledge which is assembled 
beforehand based on the specified pattern which is 
established, and which calculates the goodness of fit 
of the said potential match with respect to the said 
specified pattern. 

This image processing device of the first embodi- 
ment is distinguished by the following. The aforesaid 
specified pattern consists of a number of cells. The 
density of one or more of these cells is made to differ 
from that of those surrounding it or them. The seg- 
ment containing the information which is coded in this 
way may be a portion of the pattern or the entire pat- 
tern. 

The second embodiment concerns an image 
processing scheme ordered as follows. First, marks 
are detected which are arranged in a given spatial re- 
lationship in image data printed on, for example, bank 
notes or negotiable securities. Next, from the location 
of the said marks, a region is obtained which has a 
given spatial relationship with respect to them. The 
image data in the said region are extracted, and a de- 
termination is made as to whether they contain a pat- 
tern which tags this document as a target. 

Background Of The Invention 

The development in recent years of copying de- 
vices such as full-color copiers has enhanced the 
quality of reproduction to a level at which one cannot 
distinguish the copy from the original with the naked 
eye. Such faithful reproductions can now be obtained 
inexpensively. However, we must consider that along 
with this benefit comes the increased risk of copiers 
being put to such illicit uses as counterfeiting money, 
negotiable securities, and other original documents 
whose reproduction is legally prohibited. To avert this 
danger, various devices have been developed to pre- 
vent copiers from being used for counterfeiting. One 
such device is the image processing device disclosed 



in Japanese Patent publication 2-210481, 2-210481 
and an American patent application serial No. 
08/321,651 filed on October 11, 1994. 

That prior art image processing device reads the 

5 original image placed on the glass of the copier by 
scanning the entire original four times. It is used in 
full-color digital copy machines. Let us assume that a 
bank note has been loaded into the copy machine. On 
the first scan, the device detects the rough location 

10 of watermark B on bank note A, the document whose 
forgery is to be prevented (See Figure 68), on the as- 
sumption that the document on the glass is, in fact, a 
bank note. 

On the second scan, the device detects, based 

15 on the location of watermark B, the exact position of 
the bill and the angle at which it is placed (i.e., the ex- 
act position coordinates of the bill on the glass). Since 
the length, size and shape of bank note A, the item to 
be detected, are known in advance, when the data ex- 

20 ceed a given threshold value we convert them to bi- 
nary values and extract edges C (hatched area) of 
bank note A. Based on the location of these edges, we 
obtain the coordinates of two of the bill's apices D, (x 1( 
yO and (x 2 , y 2 ). 

25 In this way we can obtain slope 0. On the third 

scan, the device calculates the position coordinates 
( x 3. Y3> of the red seal E stamped on the bill from the 
exact position of the bill obtained in the second scan 
(i.e., from the coordinates of apices D and angle €>). 

30 Based on these position coordinates, it extracts an 
image from the region in which seal E is placed, and 
it determines whether there is in fact a red seal on this 
item. 

By scanning the item three times, the device is 

35 able to determine whether the document loaded in the 
copier is one which it would be illegal to copy (in this 
case, a bank note). If the device detects that a prohib- 
ited document such as a bank note has been loaded 
to be copied, it will perform its specified anti- 

40 counterfeiting operations on the fourth scan, such as 
making the entire display go black and preventing the 
copier from operating. 

Since existing devices such as that described 
above require a number of scans to detect a bank note 

45 or other item which cannot be copied, arriving at a 
judgment is time-consuming. Furthermore, not all 
full-color copiers use a four-scan copying scheme 
like the one described above. Some scan the original 
three times, and others only once. The processing de- 

50 vice described above cannot be used with these 
types of copiers. Another problem is that an extreme- 
ly large memory capacity is needed merely to store 
data concerning one feature such as the size of paper 
money. Inevitably, then, only a limited number of 

55 kinds of money can be detected. 

With the existing scheme, a pattern is selected 
from all the patterns on a document which may not be 
copied. This pattern must have the appropriate num- 
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ber of features so that it can be clearly distinguished, 
and it must be a pattern which cannot be found on or- 
dinary documents which may be copied. This pattern 
is matched against suspect patterns on the document 
to be copied. Even if the process is limited to a single 5 
country, there are an enormous number of docu- 
ments which may not be copied. Attempting to detect 
all of them with certainty requires a huge memory ca- 
pacity and lengthy processing time. Existing devices 
are unable to perform this processing in real time. If, 10 
after the copier is manufactured, a new document ap- 
pears which may not be copied, the device will not be 
able to handle the document or even detect it. To de- 
tect the document, a new set of features must be se- 
lected and taught to the copier, which is a complicated 15 
procedure. If we are to include the documents of other 
countries, the problem becomes even more over- 
whelming. 

Such a device requires three scans to determine 
whether the document loaded in the copier is a bank 20 
note or some other item which may not be copied. And 
if, for example, apex D of bank note A is folded or 
otherwise mutilated, the device will be unable to ob- 
tain the location of seal E. Consequently, it will be un- 
able to determine whether the document being proc- 25 
essed is actually bank note A. 

One solution to this problem is shown in Figures 
69 (A) and (B). Here guidelines F, consisting of thick 
solid or broken lines, are printed so as to enclose spe- 
cified pattern E. We extract the interior of the region 30 
enclosed by guidelines F and determine whether this 
is the specified pattern. When guidelines F like those 
in the drawing are used, it becomes easier to separate 
the specified pattern from other image data. Using 
straight lines makes it easier to calculate slope, an- 35 
gles and coordinates of apices, so the pattern can be 
extracted more simply and accurately. 

However, guidelines F are prominent enough 
(i.e., the feature is large enough) that they, and with 
them the pattern E which they designate, can easily 40 
be recognized. This makes it easier for a person to 
mutilate guidelines F or pattern E (by adding extra 
lines, erasing portions, and so on). Such mutilation 
may affect whether the pattern can be recognized 
(and so extracted), or even if the pattern is recog- 45 
nized, the device may be unable to determine wheth- 
er it is the specified pattern. 

When an existing scheme or the scheme descri- 
bed above using guidelines F is used, a pattern must 
be selected from the shapes found on the prohibited so 
document which is well suited for a characteristic 
count, such as the red seal on a bank note. (A suitable 
pattern is one which can be conclusively distinguish- 
ed and which will not be found on any ordinary docu- 
ment which may be copied.) Candidates are then 55 
matched against this pattern. Even if we limit the 
scope to a single country, there is a vast number of 
documents which may not be copied. To detect all of 



them reliably would require an extensive memory ca- 
pacity and a significant amount of processing time. 
This would make it impossible to perform the copier 
processing in real time. 

If, after the copier is manufactured, a new docu- 
ment appears which may not be copied, the device 
will not be able to handle the documentor even detect 
it. To detect the document, a new set of features must 
be selected and taught to the copier, which is a com- 
plicated procedure. If we are to include the docu- 
ments of other countries, the problem becomes even 
more overwhelming. 

Summary of The Invention 

In view of the difficulties discussed above, the 
first objective of this invention is to provide an image 
processing device which is able to detect a bank note 
or other proscribed document by scanning it only 
once; which will not require a large memory capacity; 
which will be capable of performing high-speed proc- 
essing in real time; which can be produced at a low 
cost; which can detect specified patterns with cer- 
tainty, with a very small probability of false positives; 
which can detect a broad range of documents; and 
which will be able to handle new documents which 
may not be copied, as well as a copier in which such 
a device is employed. 

The second objective of this invention is to pro- 
vide an image processing scheme and device which 
can solve these various problems as well as a copier, 
scanner and printer in which they are employed. More 
specifically, this scheme and device will be able to 
recognize unmistakably a specified pattern on a 
document and specify its location. This will make the 
extraction of the pattern and the determination of its 
identity easier and more accurate. The third objective 
of this invention is to provide a scheme and device 
which will use the minimal number of scans to detect 
and evaluate a pattern, and which will ideally be able 
to perform the processing on a single scan. Yet the 
fourth objective is to provide a scheme and device 
such that the specified pattern (including the posi- 
tioning marks, if such marks are used to pinpoint the 
location of the pattern) would be difficult for an unin- 
formed person to discern, so as to minimize the prob- 
ability of tampering. 

First Embodiment 

To achieve the objectives outlined above, the im- 
age processing device of the first embodiment is 
equipped with a device to detect a pattern in image 
data which it has received which is a potential match 
for a specified pattern; a device to extract this pat- 
tern, once detected, from the aforesaid image data; 
and a device to perform a fuzzy inference using fuzzy 
knowledge which was assembled beforehand based 
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on the specified pattern which was established, and 
which calculates the goodness of fit of the said poten- 
tial match with respect to the said specified pattern. 
The aforesaid specified pattern consists of a number 
of cells. The density of one of these cells is made to 5 
differ from that of those surrounding it. The segment 
containing the coded information may be a portion of 
the pattern or the entire pattern. 

Ideally, the aforesaid matching device creates a 
gradated image by reducing the resolution of the said 10 
image data which it has received and matches this 
gradated image against a gradated image of the said 
specified pattern which was established beforehand. 

A number of the aforesaid specified patterns may 
be established which have in common a portion out- 15 
side the aforesaid coded information and/or a portion 
of that information. The specified pattern may consist 
of a number of cells arranged either in a grid of dimen- 
sions n x m, or in concentric circles. 

To cause the density of a cell to vary from that of 20 
the surrounding cells, one could saturate the entire 
cell with a color and use its entire surface as a region 
from which to extract a characteristic count; however, 
considering the danger of misregister, it would be bet- 
ter to establish in a specified location within a given 25 
cell a fictitious mark whose diameter is less than the 
width of a cell and then vary the density of this mark. 
Alternatively, an identical result can be obtained by 
selecting as the sampling region the portion of that 
cell which is within a circle of specified location whose 30 
diameter is less than the width of the cell. 

The density of the aforesaid cell should have a 
number of grades, so that the density of a single cell 
can be expressed as multivalued data. 

The copier of this invention has at least a device 35 
to scan a document; a device which is connected to 
said scanning device and which transforms the image 
data which that device has scanned into color signals 
for printing; and a printing device, which receives the 
output of the signal transformer and performs speci- 40 
fied processing needed for printing. This copy ma- 
chine has one of the image processing devices de- 
scribed herein. It inputs the image data output by the 
aforesaid document scanner in parallel to the afore- 
said signal transformer and to the aforesaid image 45 
processor, which determines whether the said speci- 
fied pattern can be found in the document being proc- 
essed. It sends a control signal to the specified proc- 
essing device in the copy machine at least when it has 
determined that the said specified pattern is present so 
in order to prevent the copying operation from pro- 
ceeding. 

The device to detect a specified pattern reduces 
the resolution of the specified pattern which it is sup- 
posed to detect using image data which it receives to 55 
create a gradated image. It uses this gradated image 
in the matching process to detect candidates which 
are potential matches forthe specified pattern. It then 



sends the image data concerning the said candidates 
to the device which extracts the specified pattern. 
Based on the image data which it has stored, this de- 
vice extracts the image which is in the specified re- 
gion independently from the processing performed 
for the aforesaid rough search, ft sends the extracted 
image to the matching device, which performs the 
next stage of the process. This device matches the 
patterns using fuzzy inference and determines 
whether the candidate which has been found is in fact 
the specified pattern. Using fuzzy inference to com- 
pare the patterns allows us to overcome the problem 
of misregister or printing error so that proscribed 
documents will be detected reliably and accurately. 
The device to detect a specified pattern narrows the 
field of candidates before the final matching process 
is performed, so the actual number of candidates 
which must be matched is reduced. 

With this invention, there will be in at least a por- 
tion of a given specified pattern which is detected 
coded data which were determined ahead of time. Be- 
cause this specified pattern is printed somewhere on 
the bank note, stock certificate, or other item which 
may not be copied, a proscribed document will be de- 
tected accurately regardless of what type of docu- 
ment it is or what its shape or appearance may be. 
There is virtually no possibility that an ordinary docu- 
ment would have a pattern identical to a specified 
pattern in at least a portion of which there are coded 
data. This minimizes the probability of false positives. 
Since the object to be detected is already determined, 
processing can be performed at high speed. 

When a bank note or other proscribed document 
has been loaded in a copier in which the image proc- 
essing device of this invention is installed, the speci- 
fied pattern on the document will be detected. A pro- 
hibit copy command will be issued, and the item will 
not be copied or output. 

Second Embodiment 

To achieve the objectives outlined above, the im- 
age processing scheme of the second embodiment is 
designed as follows. Positioning marks, which are ar- 
ranged in a given spatial relationship, are detected 
among the image data received. From the location of 
the said positioning marks which are detected, a re- 
gion is obtained which is in a specified relationship 
with respect to the marks. The image data from the 
said region are extracted from the data received dur- 
ing a given number of scans performed after the 
marks are detected. A determination is then made as 
to whether the pattern which tags a document as one 
which may not be copied has been found among the 
said image data. 

Ideally, in addition to the said positioning marks 
in a given location on the document, there will be an 
arrangement of dummy marks which are either iden- 
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tical or similar to the positioning marks. These dummy 
marks can be used in the aforesaid specified proc- 
essing. If the specified pattern is composed of marks 
which are identical or similar to the positioning marks, 
but which are arranged in a given spatial relationship 
different from that of the positioning marks, the afore- 
said specified processing can be performed using 
these marks as a basis. 

An image processing device according to this in- 
vention in which such a scheme could be implement- 
ed consists of a device to detect among the image 
data which it receives a specified set of positioning 
marks provided in given locations on documents to be 
detected; a device to extract the region containing the 
specified pattern, which it calculates using data con- 
cerning the positions of the marks detected by the 
said detection device; and a device to compare two 
patterns, which determines whether the specified 
pattern which occurs on documents to be detected 
can be found in the region extracted by the extraction 
device. 

Ideally, the said device to detect the positioning 
marks detects at least three positioning marks which 
are arranged in a predetermined spatial relationship. 
The shape formed by the imaginary lines linking the 
said three or more adjacent marks arranged in a spe- 
cified relationship is an isosceles triangle or some 
other asymmetrical shape. 

The device to detect the positioning marks should 
have a device to convert to binary form the image 
data it receives and a searching device which uses a 
mask partitioned into n x m units to detect, from the 
binary dark-light image data detected in each cell 
comprising the said mask, any mark identical to the 
said positioning marks, and detect its coordinate 
data. Then, despite its simple configuration, the de- 
vice will be able to process data at high speed. 

If the said device to compare two patterns ob- 
tains the degree of resemblance between the said 
pattern in the extracted region and the reference pat- 
tern, or more specifically, if it uses fuzzy inference 
(i.e., fuzzy pattern matching) to calculate the degree 
of resemblance, the device will cope well with shifts 
of position, and its recognition rate will be high. 

Alternatively, the said device to compare two pat- 
terns may, for example, partition the said extracted 
region into N cells; decode it using the result obtained 
for the density of each cell; and determine whether 
the decoded data match the coded reference data on 
the document to be detected, which were obtained 
earlier. 

Another processing scheme which would achieve 
the objectives stated above would be to detect, 
among the image data received, a given set of iden- 
tical marks, store the location of these marks, and ob- 
tain the number of marks on the outline of a given 
shape based on data concerning the location of the 
said marks. From the total number of these marks, it 



can be determined whether the pattern detected 
among the said data is, in fact, the specified pattern. 

When this scheme is used, the various proce- 
dures described above could also be performed on an 

5 arrangement of dummy marks which are identical or 
similar to the marks in specified locations on the 
document, as described earlier. If the dummy marks 
are identical to the real marks, they must be located 
away from the outline of the shape delineated by the 

10 real marks constituting the specified pattern. If the 
dummy marks merely resemble the real marks, they 
may be placed anywhere. It is also desirable that one 
more identical mark be placed in the center of a spe- 
cified pattern consisting of a number of identical 

15 marks arranged along the outline of a given shape. 

The relative location of the said marks on the out- 
line of the given shape can serve as a criterion for 
concluding that a specified pattern consisting of a 
number of the said marks has been found, so that a 

20 determination as to whether the pattern which would 
tag the document can be found in the said image data 
is based on the total number of marks in the outline 
of the said shape and on data concerning the location 
of each mark in the outline of the given shape. 

25 The device in which the various schemes dis- 

cussed above is implemented may, for example, con- 
sist of a device to detect among the image data which 
it receives specified marks provided in given loca- 
tions on documents to be detected: and a device 

30 which receives data concerning the locations of the 
marks detected by the said detection device, and, us- 
ing at least the criterion of whether there is a specific 
number of marks along the outline of a given shape, 
determines whether the specified pattern has been 

35 found. The device to detect the marks may have es- 
sentially the same configuration as the device to de- 
tect positioning marks described above. 

It is desirable that the said device to detect a spe- 
cified pattern should weight the data with a coeffi- 

40 cient which decreases with the distance from the out- 
fine of the said shape, and consider both the number 
of marks detected and the coefficients of the loca- 
tions of said marks when determining whether this is 
the specified pattern. The said device to detect a spe- 

45 cif ied pattern should at least be equipped with a de- 
vice which calculates the goodness of fit of a pattern 
based on the number of marks detected, and a device 
which, when the goodness of fit calculated by the said 
calculating device for a certain pattern exceeds a 

so specified value, matches that pattern, using rotary 
matching, against at least one previously provided 
reference pattern to detect the desired specified pat- 
tern. 

The copier, scanner and printer of this invention 
55 are equipped with one of the image processing devic- 
es described above. The image data output by the de- 
vice in the copier which reads the document are input 
in parallel fashion into both the device to convert color 
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signals and the said image processing device. In- 
put/output signals to the control device in the scanner 
or printer are input in parallel to the image processing 
device. 

Based on specific image data obtained by scan- 5 
ning the document, the device performs processing to 
detect the said specified pattern. It determines 
whether or not the said pattern can be found on the 
document loaded for copying. If it determines that the 
said pattern is present, it sends a control signal to the w 
said processing device to control the specified proc- 
essing, which is either copying, scanning the image, 
or outputting the copy (printing it out). 

The device to detect the positioning marks is op- 
erated. An inspection is made to determine whether 15 
there are positioning marks among the image data re- 
ceived. If such marks are detected, data representing 
their position coordinates are obtained. The device to 
extract the specified pattern is operated. From the lo- 
cation of the positioning marks, the location of the 20 
specified pattern, which is in a given spatial relation- 
ship with respect to the marks, is calculated. The im- 
age in a given region is extracted based on its loca- 
tion among the image data received. (These data are 
the same ones in which the positioning marks are de- 25 
tected.) The extracted data are sent to the device to 
compare two patterns, where the next stage of the 
operation will be performed. The given comparison is 
performed and it is determined whether the specified 
pattern can be found in the given region. The device 30 
to detect a specified pattern narrows the field of can- 
didates before the final matching process is per- 
formed, so the actual number of candidates which 
must be compared is reduced, and the determination 
can be made in real time. 35 

When the specified pattern consists of a number 
of identical marks arranged in a given spatial relation- 
ship without the use of positioning marks, the proc- 
essing performed by the device to detect the marks 
is virtually identical to that used to detect the said pos- 40 
itioning marks. The specified pattern consists of a 
given number of marks arranged to form the outline 
of a given shape. On this basis, the device to detect 
the specified pattern can count the marks in the out- 
line of a given shape. By determining whether the val- 45 
ue it finds is a specified number (with a margin pro- 
vided above and below it), the device can determine, 
in a simple fashion, whether the specified pattern has 
been found. 

If more than one specified number is used as a so 
judgment criterion, the value of the pattern consisting 
of detected marks is compared with a number of ref- 
erence values sequentially. Based on each of these 
values, a determination is made as to whether the 
specified pattern has been found. In this case, coded 55 
data can be added to the pattern, detection can be 
made more complex, and mistaken identification can 
be minimized, thus enhancing the reliability of the de- 



vice. 

If the data are weighted according to their loca- 
tion, and given a coefficient which decreases with the 
distance from a given location, then a given mark will 
be counted among the total even if it has been shifted 
slightly from its predicted location as a result of a mis- 
print or detection error. This will enable the specified 
pattern to be detected conclusively, without mistakes 
in identification or detection. 

If the location (i.e., the relative location) of each 
mark in the outline of a given shape is also used as 
a criterion for detecting the specified pattern, the 
processing is as follows. First, the goodness of fit of 
the pattern is calculated by the device for that pur- 
pose, based on the number of marks detected. Next, 
if the goodness of fit calculated by the said device ex- 
ceeds a fixed value, the matching device matches the 
pattern against at least one previously established 
reference pattern and determines if the patterns 
match. If the specified pattern contains angle data, 
determinations can be processed more rapidly. If, 
without considering angle data, candidates are pre- 
processed (eliminated) according to their goodness 
of fit, the number of candidates which must actually 
be matched will be much reduced, and processing 
can be performed at high speed. 

As long as the coordinates of the positioning 
marks or those constituting the specified pattern are 
specified, the marks can be reduced in size to make 
them less conspicuous. This will make it more difficult 
for an uninformed person to find the pattern, which is 
in a given spatial relationship with respect to the pos- 
itioning marks. So if any person attempts to counter- 
feit a bill by tampering with a specified part of it so that 
the positioning marks or the pattern itself will not be 
detected, he will not be able to pinpoint the location 
of the marks. Bills marked in this way will be difficult 
to tamper with; even if they are tampered with, there 
is little likelihood that the tampering will correspond 
to the location of the marks. Prohibited documents 
such as bank notes will be detected with absolute cer- 
tainty. If dummy marks are provided in addition to the 
positioning marks, tampering can be suppressed 
even more effectively. 

When a bank note or other proscribed document 
has been loaded in a copier in which the image proc- 
essing device of this invention is installed or when a 
scanner and printer are caused to scan the document 
and print it out, the specified pattern on the document 
will be detected. A halt copy command will be issued, 
and the item will not be copied, scanned, or printed 
out. 

Brief Description Of The Invention 

Figure 1 shows the overall configuration of an 
ideal first and second embodiments of the image 
processing device of this invention. 
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Figure 2 is a block diagram of the internal struc- 
ture of the device to detect a specified pattern. 

Figure 3 is a block diagram of the internal struc- 
ture of the device to perform the averaging operation. 

Figure 4 is a block diagram of the internal struc- 
ture of the flip-flop groups in the detection device. 

Figure 5 is a block diagram of the internal struc- 
ture of the decoder circuit in the detection device. 

Figure 6 illustrates how the preliminary judgment 
unit works. 

Figure 7 is a block diagram of the internal struc- 
ture of the device to extract a specified pattern and 
the device to match it against a reference pattern. 

Figure 8 shows how corner detection works in the 
position detection unit. 

Figure 9 is a flow chart illustrating the function of 
corner detection in the position detection unit. 

Figure 1 0 shows how mark detection works in the 
position detection unit. 

Figure 11 is a flow chart illustrating the function 
of mark detection in the position detection unit. 

Figure 12 is a flow chart illustrating how the de- 
vice to match a specified pattern against a reference 
pattern works. 

Figure 13 gives an example of a specified pattern 
used in this first embodiment 

Figure 14 (A) illustrates the advantage conferred 
by the specified pattern used in this first embodi- 
ment. 

Figure 14 (B) shows a modified version of this 
specified pattern. 

Figure 15 shows another version of the specified 
pattern. 

Figure 16 illustrates the operation of the device. 

Figure 17 gives the membership function of each 
cell in the pattern shown in Figure 16. 

Figure 18 illustrates another way in which the 
specified pattern may be employed. 

Figure 19 is a flowchart showing some of the op- 
erations and functions performed by the device to 
match a specified pattern when one of the patterns 
shown in Figure 18 is used. 

Figure 20 shows another version of the specified 
pattern. 

Figure 21 shows another version of the specified 
pattern. 

Figure 22 shows another version of the specified 
pattern. 

Figure 23 shows another version of the specified 
pattern. 

Figure 24 shows another version of the specified 
pattern. 

Figure 25 shows another version of the specified 
pattern. 

Figure 26 shows an example of the copy machine 
with which this invention is concerned. 

Figure 27 shows an example of the copy machine 
with which this invention is concerned. 



Figure 28 shows an example of positioning 
marks, dummy marks and a specified pattern which 
might be used with this invention. 

Figure 29 shows another example of positioning 
5 marks, dummy marks and a specified pattern which 
might be used with this invention. 

Figure 30 shows another example of positioning 
marks, dummy marks and a specified pattern which 
might be used with this invention. 
10 Figure 31 is a block diagram of the internal struc- 

ture of a device to detect the positioning marks. 

Figure 32 is a block diagram of the internal struc- 
ture of the flip-flop group in the detection device. 

Figure 33 is a block diagram of the internal struc- 
15 ture of the decoder circuit in the detection device. 

Figure 34 gives an example of dot patterns which 
might be used for positioning marks and dummy 
marks. 

Figure 35 is a block diagram showing the internal 
20 organization of the device to extract the specified 
pattern and the device to match this pattern against 
a reference pattern. 

Figure 36 illustrates the operation of the device 
to extract the specified pattern. 
25 Figure 37 is a flowchart of the functions per- 

formed by the control CPU which constitutes one por- 
tion of the device to extract the specified pattern. 

Figure 38 illustrates the operation of the device 
to match the specified pattern against a reference 
30 pattern. 

Figure 39 shows examples of the rules and mem- 
bership functions stored in the rule memory and MF 
memory in the device to match the specified pattern 
against a reference pattern. 
35 Figure 40 shows an example of a binary conver- 

ter circuit used to detect positioning marks when the 
positioning marks and the dummy marks are distin- 
guished by having different colors. 

Figure 41 shows another example of positioning 
40 marks and dummy marks. 

Figure 42 shows another example of a specified 
pattern. 

Figure 43 shows the overall configuration of an- 
other ideal example of the image processing device 
45 of this invention. 

Figure 44 shows an example of a specified pat- 
tern used in this second embodiment. 

Figure 45 is a block diagram of the internal struc- 
ture of the device to detect the marks. 
so Figure 46 illustrates the operation of the device 

to detect the marks. 

Figure 47 is a block diagram of the internal struc- 
ture of the device to detect the specified pattern. 

Figure 48 is a flowchart of the functions per- 
55 formed by the CPU. 

Figure 49 illustrates the operation of the device 
to detect the specified pattern. 

Figure 50 illustrates the operation of the device 
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to detect the specified pattern. 

Figure 51 illustrates the operation of the device 
to detect the specified pattern. 

Figure 52 illustrates the operation of the device 
to detect the specified pattern. 5 

Figure 53 is a block diagram of the internal struc- 
ture of the device to detect the specified pattern. 

Figure 54 shows the internal structure of the 
combination circuit. 

Figure 55 shows another example of a specified 10 
pattern. 

Figure 56 shows another example of a specified 
pattern. 

Figure 57 is a block diagram of another internal 
structure for the device to detect the specified pat- 15 
tern. 

Figure 58 shows an example of how code data 
can be incorporated in the specified pattern. 

Figure 59 shows an example of how code data 
can be incorporated in the specified pattern. 20 

Figure 60 shows an example of how code data 
can be incorporated in the specified pattern. 

Figure 61 shows a block diagram of the essential 
internal structure of a device to detect a specified 
pattern which can detect the patterns with code data 25 
shown in Figures 35 through 37. 

Figure 62 shows another example of how code 
data can be incorporated in the specified pattern. 

Figure 63 shows another example of how code 
data can be incorporated in the specified pattern. 30 

Figure 64 shows another example of how code 
data can be incorporated in the specified pattern. 

Figure 65 is a block diagram of the essential in- 
ternal structure of a device to detect a specified pat- 
tern which can detect the patterns with code data 35 
shown in Figures 39 through 41. 

Figure 66 shows an example of a scanner accord- 
ing to this invention. 

Figure 67 shows an example of a printer accord- 
ing to this invention. 40 

Figure 68 illustrates the operation of an image 
processing device belonging to the prior art. 

Figure 69 illustrates one idea used in the prior art 
to solve a problem. 

45 

Detailed Description Of The Invention 

First Embodiment 

We shall next give a detailed explanation, with 50 
reference to the appended drawings, of a first ideal 
embodiment of the image processing device and sys- 
tem and the copier in which these are used which are 
designed according to this invention. The processing 
device in this example is installed in a full-color copy 55 
machine. It is designed to detect paper money when 
someone attempts to copy it and prevent the copying 
process from proceeding to completion. 



The substance of this invention is that a specified 
pattern (details of which will be discussed later) which 
can be detected reliably, is unlikely to be found on any 
ordinary document, and is unlikely to give rise to false 
positives is prepared beforehand. This pattern is 
printed in a desired location on bank notes, stock cer- 
tificates, or any other documents which may not be 
copied. The image processing device is given knowl- 
edge of this specified pattern. When a new document 
appears which may not be copied, the specified pat- 
tern can be put on it, and it too will be detected. Since 
the item to be detected is always the same specified 
pattern, it can be applied to a wide variety of proscri- 
bed documents. Processing can be performed at high 
speed. 

As can be seen in Figure 1, the image data which 
are scanned by the image sensor in the copier are in- 
put into detection device 1, the device which detects 
the specified pattern. As these image data are scan- 
ned by an image sensor such as a CCD, they are 
transmitted successively, region by region, to device 
1. The actual data which are sent are full color data 
for each of the red (R), green (G) and blue (B) com- 
ponents, with resolution of 400DPI. 

Detection device 1 compresses the image data it 
has received to form a gradated image. Using masks 
of specified sizes, it checks the area of this image for 
patterns which are possible matches with each spe- 
cified pattern which is to be detected (a mark, figure, 
etc.). If it detects a possible candidate, it roughly spe- 
cifies a reference position (the center for a circle, the 
apex of a corner for a rectangle, and so on) for it and 
transmits the relevant data to device 2, the device to 
extract a specified pattern. 

Extraction device 2 detects these reference pos- 
itions more precisely, based on the image data before 
they are compressed. These data include the poten- 
tial matches detected during the rough search. De- 
vice 2 also creates patterns to be used for compari- 
son in the pattern matching process to be performed 
later. Potential matches are stored temporarily in a 
specified storage area in device 2. Extraction device 
2 makes a preliminary judgment as to whether the ex- 
tracted patterns resemble each specified pattern to 
be detected, based on the detailed, non-compressed 
image. If it finds a pattern similar to a specified pat- 
tern, it extracts the portion of it which surrounds the 
aforesaid reference position and transmits this por- 
tion to device 3, where it will be matched against the 
specified pattern to be detected. 

Device 3 uses fuzzy pattern matching logic to de- 
termine the goodness of fit between each extracted 
pattern and its reference pattern (i.e., the specified 
pattern to be detected), and it judges whether this 
pattern is, in fact, the one which is being sought If it 
is, device 3 outputs a specified control signal. 

In this example, then, a rough search is made of 
the data scanned by the copier in its normal fashion. 
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In this search, the gradated data are matched one by 
one against a pattern which is synthesized from the 
aforesaid reference pattern, and potential candidates 
for a match are extracted. The data concerning these 
potential matches are stored in a storage area, and 5 
they are matched against the reference pattern by 
means of fuzzy inferences when the specified proc- 
essing sequence has been completed. Each potential 
match is judged as to whether it is, in fact, the pattern 
being sought. Matching device 3 uses for comparative 10 
purposes a detailed image which approximates the 
original image. Before an attempt is made to match 
candidates to this detailed image, their number has 
been reduced by conducting a rough search at high 
speed. (Candidates can be evaluated swiftly and ac- 15 
curately when judgments are based on gradated im- 
age data.) All processing is performed at high speed 
in real time. All data concerning candidates for the 
matching procedure are stored temporarily in the ex- 
traction device. Even though image data may be sent 20 
to detection device 1 successively, device 1 performs 
the detection processing without regard to the state of 
the latter-stage processing (i.e., without waiting for 
the results of that processing), and the processing 
can be performed in real time at high speed. 25 

In this section we shall discuss the actual config- 
uration of each device referred to above. Pattern de- 
tection device 1, which detects a specified pattern, is 
pictured as pattern detection device 1* in Figure 2. 
Here we see that the image data are sent to spatial 30 
averaging unit 10, where a specified number of neigh- 
boring pixels in the relevant data are combined to 
form larger units. The density of the pixels in each unit 
is averaged to create a gradated image consisting, as 
it are, of rather larger pixels. 35 

Averaging unit 10 might consist of the compo- 
nents shown in the block diagram in Figure 3. Each 
output from latches 11a through 11 d, which are con- 
nected in four stages that sequentially output data 
synchronized by a shift clock, is input into adder 1 2a. 40 
Each pixel in each line of the image data is input, in 
sequential order, into latch 11a. Each pixel is repre- 
sented as multivalued data which express its density 
(or gradation of brightness). Thus in each of latches 
11a through 11d, a specified number of bits is set 45 
which correspond to these multivalued data. 

The first adder, 12a, adds the four values which 
represent the densities of the pixels stored in latches 
Ha through 11d, which are the last four pixels to have 
been input (the last four pixels on the path which is be- 50 
ing scanned), and transmits the result to the second 
adder, 12b, through one of its input terminals. The 
sum of the densities of specified pixels in the last 
three lines, which are stored in line memory 13, is in- 
put into the other input terminal of adder 1 2b via AND 55 
circuit 14. The result of the addition performed by ad- 
der 12b is sent to a specified address in the aforesaid 
line memory 13 (i.e., the address which is read out in 



order for adder 12b to perform its addition) and to div- 
ider 15. 

The output of adder 12b is written into line mem- 
ory 13 each time four pixels have been added, in re- 
sponse to a control signal which is not pictured in the 
diagram. In other words, the first through fourth pixels 
on the line being processed are input simultaneously 
into adder 12a. Based on the result of the addition 
performed by adder 12a, further addition is per- 
formed by adder 12b, after which the first write-in of 
data into line memory 1 3 is performed. When the fifth 
through eighth pixels have been added, the result is 
input into line memory 13 in the second write-in. This 
procedure is performed repeatedly. In this way, the 
result of the addition of every four pixels on line 1 is 
stored in order in line memory 13. 

When line 2 is processed, the result of adding ev- 
ery four pixels is obtained from latches Ha through lid 
and adder 12a just as with line 1; at the same time, 
however, previous results of addition which are stored 
in line memory 13 are read out, and adder 12b adds 
the results obtained by adding every four pixels on 
line 2 to the corresponding results for every four pix- 
els on line 1. In this way the densities of eight pixels, 
four of which are contiguous to the other four in each 
of two rows, are added together; the result of this ad- 
dition is stored in line memory 13. 

When line 3 (or line 4) is processed, the totals of 
the densities of certain pixels in lines 1 and 2 (or lines 
1 through 3), which are obtained by the addition per- 
formed during the processing of the previous lines 
and stored in line memory 13, are added to the cor- 
responding densities in the current line. When line 4 
is processed, the result of the addition performed by 
adder 12b (the total of the densities of 4 x 4, or 16, pix- 
els) is sent to divider 1 5, where it is divided by 1 6 to 
obtain an average value. The control signal which is 
input into AND circuit 15 goes to zero once line 4 has 
been processed. Readout of data from line memory 
13 is prohibited and the cumulated value is reset. 

By increasing or decreasing the number of latch- 
es, we can increase or decrease the number of pixels 
whose density is averaged. Increasing the number of 
latches will produce an image with lower resolution. 
By varying the number of latches and choosing an ap- 
propriate timing for the input of the control signal (0) 
to AND element 14, we can obtain an average density 
for N x M pixels. Which conditions are appropriate will 
depend on the image to be matched. 

The average density of each sixteen pixels is 
used to construct an image consisting of new, larger 
pixel units. The data concerning this new image are 
sent to binary processor 1 6, where at a certain thresh- 
old density they are converted to binary values. The 
resulting binary image is stored temporarily in line 
memory 1 7. This generated image is more blurry than 
the original image. Details of the original patterns in 
the image have been lost and only rough shapes can 
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be discerned. By setting an appropriate threshold for 
the conversion to binary, we can produce an image 
in which the interior of a given pattern will appear to- 
tally black while the background is totally white. Even 
if a misregister or other printing error in the original 5 
image as read out and input results in a slight discrep- 
ancy between the specified pattern which is stored 
and the pattern which is being checked, the discrep- 
ancy will vanish when the image is gradated. This 
scheme allows the image to be detected simply and 10 
quickly. 

The output from the aforesaid binary processor 
1 6 is sent to search unit 1 8, where the gradated image 
with low resolution (actually, the binary data) is 
searched for a pattern of a given shape. That is to say, 1 5 
a search is made for a specified pattern in a region 
of a given size, such as a mask with dimensions N x 
N. This search will be explained in concrete fashion 
below. 

Search unit 18, shown in Figure 2, is composed 20 
of flip-flop group 18a; circuit to output coordinates 
18b, which receives the output from flip-flop group 
1 8a, obtains the point coordinates of the hits which in- 
dicate the positions of patterns representing possible 
matches, and outputs those coordinates; decoder cir- 25 
cuit 18c ( which specifies the orientation and other 
features of the patterns; and line memory 18d, in 
which are stored the data representing detected pat- 
terns which are possible matches. 

The actual arrangement of flip-flop group 18a is 30 
shown in Figure 4. In this example, the area being 
processed consists of 3 pixels of scanning x 3 pixels 
of feed, or 9 (3 x 3) flip-flops FF. The three pixels W10, 
W1 1 and W1 2, which are in line in the feed orientation, 
are taken from the image data stored in line memory 3S 
17, synchronized by means of the shift clock SCLK, 
and input to the first column of flip-flops, F00, F1 0 and 
F20. New pixel data are input to W10, W11 and W12 
sequentially according to timing provided by the shift 
clock, and the data are transmitted to the next column 40 
of flip-flops. The pixel data which are input for the bi- 
nary image, Q00, Q01, Q02, Q10, Q11, Q12, Q20, 
Q21 and Q22, are output through the output terminal 
Q on each flip-flop. For a black pixel, the output of the 
flip-flop will be "1". 45 

When all scanning data up to the end of one line 
have been input, the device shifts down one line in 
the direction of feed and inputs data from the begin- 
ning of that line. In this way, the 3x3 pixel area (the 
mask) which is the domain of the search is moved so 
over the surface of the document. 

Each of the aforesaid outputs, Q00, Q01, Q02, 
Q10\ Q11, Q12, Q20, Q21 and Q22, is input into de- 
coder circuit 18c. Decoder circuit 18c is pictured in 
Figure 5. If the pattern of 3 x 3 pixels which is output 55 
by flip-flop group 1 8a is one of the eight patterns (Hit 
0 to Hit 7), the output of the corresponding AND ele- 
ment will go high. If the output from decoder circuit 



1 8c for Hit 0 goes high, we know that there is a dark 
area in the top left-hand corner of the grid, centered 
around flip-flop FF11 . If the output for Hit 4 goes high, 
we know that there is a dark area covering the upper 
part of the grid, centered around flip-flop FF11. In this 
example, the object of the search is a corner. There- 
fore, patterns other than these eight, such as when 
the central pixel is in the middle of a border or entirely 
within or outside a dark area, will not be detected, and 
the output in any of these cases will go low. 

Circuit 18b is connected to the addresses in mul- 
tiple line memory 17. It outputs to flip-flop group 18a 
the addresses of images which must be input, and it 
outputs to line memory 18d the coordinates (XY) of 
the pixels output by FF1 1 , the central flip-flop in group 
1 8a. The output of decoder circuit 1 8c and the central 
coordinates XY at that moment are stored in line 
memory 18d. The detection results for several lines 
worth of data are also stored in memory 18d. 

In this example, nine flip-flops representing 3 x 3 
pixels are used to detect the corner of a figure as 
shown in Figure 4 and the states of the decoder circuit 
are those shown in Figure 5. However, the number of 
flip-flops used and the conditions for the decoder 
must be selected according to the shape of the pat- 
tern one wishes to detect and which portion of it one 
chooses to focus on. It would, of course, be possible 
to detect a circle or some other shape as required. 

In the example described above, the search for 
the specified pattern to be detected is carried out by 
hardware. It would also be possible to handle this 
processing via specified software. Line memory 18d 
in a search unit 18 which is comprised of hardware 
contains numerous unnecessary data which will not 
need to be sent to pattern extraction device 2 for the 
next step of processing. The data which are stored in 
line memory 1 8d are sent instead to preliminary judg- 
ment unit 19, where a judgment is rendered as to 
whether those data should be processed by pattern 
extraction device 2 (preselection of hit points). 

In this example, a rough search is performed with 
reference to gradated data. This allows image data 
which potentially have the specified pattern to be 
identified swiftly and surely as possible matches. At 
the same time, a large quantity of useless data are 
also detected. When the states of the pixels on the 
periphery lead to the judgment that a given pattern is 
not the specified pattern, that pattern can be elimin- 
ated without being output, obviating the need for sub- 
sequent more detailed judgments. Only the actual hit 
points are output to candidates memory 21 in pattern 
extraction device 2. 

To process this judgment, data stored in memory 
1 8d for which the output goes high (the hit points) are 
extracted, and effective hit points are detected based 
on the relative spatial relationships among these hit 
points and other hit points which are on their periph- 
ery. The position coordinates XY and the orientation 
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data for these points, HIT 0 to 7, are stored in candi- 
dates memory 21. 

To give one example, depending on the type of 
pattern being sought (its shape, size, etc.), if there is 
another hit point in the vicinity of a hit point which has 
been detected, or within a specified area around that 
hit point, then the first hit point will be regarded as val- 
id. If a number of hit points are found within a speci- 
fied area, then an appropriate decision will be made, 
depending on the pattern, to consider only the periph- 
ery (Figure 6 (A)) or only the center (Figure 6 (B)). In 
particular, if the hit points are close to one another, 
there is a high probability that the detected pattern is 
identical to the specified pattern. In this case it would 
be acceptable to try to match the patterns based on 
one of these hit points. 

Figure 7 is a block diagram illustrating the config- 
uration of pattern extraction device 2. As shown in the 
figure, device 2' continuously stores in detailed mem- 
ory 20 the RGB data (the variable density image) 
from detection device V which have not yet been con- 
verted to binary values. Memory 20 has a sufficient 
capacity to allow it to maintain storage of image data 
from the pixels (or the line) currently scanned by the 
image sensor to a specified earlier point (at least as 
large as the pattern to be detected). If its capacity is 
reached, the oldest data (those acquired earliest) are 
erased and new data are written in their place. 

As was described earlier, the coordinate values 
and orientation of valid hit points output by prelimin- 
ary judgment unit 19 in detection device V are written 
into candidates memory 21. The stored coordinate 
values are read out by control CPU 30. This CPU 
sends various control signals to extraction device 2* , 
thus controlling its operation. One such control signal 
is the command sent to detailed memory 20 telling it 
to output, based on the coordinate values read out 
from memory 21, the image data which contain the 
pattern associated with a valid hit point 

Based on the coordinates read out by CPU 30, 
the corresponding image data stored in memory 20 
are sent to binary processing unit 22. Here the RGB 
image data are converted to binary values; however, 
the threshold value for this conversion need not be 
the same as that used by binary processing unit 16 
in the aforesaid pattern detection device 1\ 

The binary data produced in this fashion are stor- 
ed temporarily in detailed memory 24 and sent in or- 
der to position detection unit (CPU) 25. Here refer- 
ence points are obtained for the matching operation 
to be performed in matching device 3. Based on data 
stored in program ROM 25a, which is connected to 
CPU 25, a certain specified position in the specified 
pattern to be detected is obtained. For a corner, this 
position is the accurate location of the apex, and for 
a mark, the accurate location of the center. If when 
position detection unit 25 has extracted an accurate 
location and it is determined that this is not the spe- 



cified pattern, output will cease. Afull explanation will 
be given below. 

When Searching for a Corner 

5 

In the rough search, the approximate apex of the 
corner (the hit point) is known, as well as the orienta- 
tion of the corner. As illustrated in Figures 8 and 9, a 
hypothetical origin 0 is established in a specified pos- 

10 ition in the vicinity of hit point H and inside the corner. 
From this hypothetical origin 0, a horizontal X axis and 
a vertical Y axis are established (Steps 1 and 2). 

Next, the necessary constant is established for 
detection of a specified position in the pattern (Step 

15 3). Moving along the X axis in steps of a specified 
width (indicated by arrows in the drawing), we scan at 
each step in a positive (upward) and negative (down- 
ward) direction parallel to the Y axis, judging whether 
the pixels are black or white. We find the border be- 

20 tween black and white (the border where black ends 
and white begins will be called an edge hereafter) 
(Steps 4 and 5), and we draw a straight line following 
the edge which has been detected as closely as pos- 
sible (segments indicated by hatching). The minimum 

25 mean square criterion, for example, might be used to 
cause the various edges which have been detected to 
approximate a straight line (Steps 6 and 7). When the 
relevant processing is performed, the two edges 
which extend on both sides of the Y axis are regula- 

30 rized as straight lines and L 2 . The intersection of 
these two lines is found and labeled apex P (Steps 8 
and 9). 

The precise orientation of the corner can be com- 
puted from the slope of one of the aforesaid straight 

35 lines (or edges) which have been detected or from the 
slope of the line bisecting the angle formed by the two 
straight lines (Step 10). The coordinates of the apex 
and the slope are transmitted to control CPU 30. If 
when the aforesaid apex is extracted two edges can- 

40 not be found, or if an insufficient number of edge data 
are found, or if the angle formed by the two edge lines 
varies substantially from the specified angle, then a 
judgment will be rendered that the pattern found is not 
the corner we are seeking (i.e., it is not the specified 

45 pattern to be detected), and output will cease. 

When Searching for a Mark 

As is illustrated in Figures 10 and 11, the hit point 
so discovered in the rough search for a mark is consid- 
ered to be the hypothetical center point 0. Lines L3 
through L6 are drawn in the four compass directions 
from center point O to form X and Y axes (Step 11). 
The necessary constants are established for the de- 
55 tection of specified positions in the pattern (Step 12). 
Since the radius of the mark is already known, the ap- 
proximate location of its circumference (i.e., its edge) 
can be determined. Aline U is drawn orthogonal to the 

12 
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radius, and this line is used to establish a window (i.e., 
an area) with a width a and a depth b (Steps 12 to 14). 

A histogram is drawn of the black pixels in the di- 
rection of the search (the radial direction) and the or- 
thogonal direction. A certain height of the histogram 
H th is chosen to be the threshold value. Levels greater 
than height H th are considered to represent the mark 
as far as they continue. The point where they end is 
considered to be the start of the blank area. The bor- 
der between these areas is considered to be the 
edge. Rather than making a judgment based solely on 
processing involving a threshold value, it would alter- 
natively be possible to extract the border between the 
pattern and non-pattern portions based on the varia- 
tion in the height of the histogram and its continuity 
(Steps 15 and 16). 

The search for the edge is performed in a radial 
direction on both the left and right sides, and a center 
position P is obtained for the edges which have been 
extracted as a result of the search. In the same way, 
the upper and lower edges are found, and center pos- 
itions are obtained for them (Steps 17 and 18). 

When only one edge can be extracted, the radius 
of the target image can be used to determine the cen- 
ter. This is done by moving back from the edge which 
has been detected along a radius a distance equal to 
the radius of the target and declaring that point to be 
the center. When the center position and the radius 
have been determined in this way, the relevant data 
are output to control CPU 30 (Step 19). 

If a center cannot be found when a number of hy- 
pothetical center points have been detected as de- 
scribed above (because the diameters vary too much 
or edges cannot be detected on both sides of the di- 
ameter), a judgment will be rendered that the image 
data being processed do not contain the specified 
pattern to be detected, and output will cease. 

Since in this example only the data within the win- 
dow are searched, the processing can be done at a 
high speed. If there is a specific shape within the spe- 
cified pattern or a characteristic of some kind inside 
the mark, the data can be expressed as a four- 
directional histogram in the same way as is described 
above. Since the angle of rotation is known, it, too, 
can be transmitted to CPU 30 as needed. 

We shall next discuss the workings of pattern 
matching device 3. As is shown in Figure 7, the coor- 
dinate data for the apex and slope of a corner resem- 
bling the specified pattern or for the center position 
of a mark are sent from detection unit 25 to CPU 30. 
Based on the data stored in program ROM 30a, CPU 
30 specifies which of the image data stored in de- 
tailed memory 24 must be read out, and it decides 
which area of the image data should be matched 
against the specified pattern to be detected. The rel- 
evant image data in this region are transmitted from 
memory 24 to fuzzy inference unit 35 based on a con- 
trol signal from CPU 30. Inferences are processed us- 



ing the fuzzy knowledge, which includes the rules 
stored in rule memory 36 and the membership func- 
tions stored in MF memory 37. A judgment is then ren- 
dered as to the degree of resemblance between the 
5 received image data and the previously established 
specified pattern to be detected (See Figure 12). 

Put more concretely, in the example of Figure 1 3, 
a round mark 41, whose diameter is less than the 
length of an edge of a cell 40, is provided in each of 

10 nine square cells, which are arranged in the 3x3 grid 
pattern, to form the pattern to be detected. The dark- 
ness of at least some (41a) of marks 41 is varied from 
that of the others (41b) to create a pattern containing 
bit data. As can be understood from the drawing, in 

is this example the dark marks 41a are completely col- 
ored, while the light marks 41b are completely uncol- 
ored. In this way each cell 40 can be made to express 
1 bit of data as it is either *'1 " or "0". Since in this ex- 
ample the overall pattern is a square, the processing 

20 to extract hit points which is performed by detection 
device 1 and extraction device 2 will use a search for 
a corner. By providing in this fashion a number of dark 
marks 41a (in cells) to which color has been applied, 
we create a pattern which will not be affected by print- 

25 ing misregisters, dirt on the document, or, when the 
pattern is extracted, errors or tampering. 

Since the relevant pattern would be printed 
ahead of time in a specified location on any docu- 
ments which may not be copied, such as bank notes 

30 or negotiable securities, the fuzzy knowledge con- 
cerning this pattern can be stored in memories 36 and 
37. 

In this example the diameter of marks 41 is less 
than the length of cells 40. Even if the orientation of 

35 the pattern is momentarily shifted by angle €>, as 
shown in Figure 14(A), marks 41 will not extend far 
enough beyond the square of their own cell to overlap 
a neighboring cell. If the angle is shifted due to a print- 
ing or scanning error or an error which occurs during 

40 image processing, there is only a minimal probability 
that there will be an error in recognition. This scheme 
also responds well when the document is moved eith- 
er horizontally or vertically, regardless of angular 
shift. 

45 In this example, the diameter of a mark 41 is less 

than the length of one side of a cell 40; however, the 
invention is not limited to this design. The diameter 
could as well be equal to the length of a side. In this 
case, only a small portion of the mark would overlap 

so the adjacent cell, so the probability of recognition er- 
rors resulting from angular shift would be small. It 
would also be acceptable to color the entire area of a 
cell 40, as shown in Figure 14(B). In this case, the rate 
of recognition when the image is shifted over a small 

55 angle, as described above, will be lower than with the 
scheme employed in this example. However, this can 
be compensated for by setting up appropriate mem- 
bership functions. 

13 
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In this example, it is desirable to provide a guide 
on the periphery of the specified pattern, as shown 
in Figure 1 5. This will facilitate the search for a corner 
which is described above, and allow us to determine 
the position and orientation of the specified pattern 5 
and of each cell. We might, for example, provide the 
guidelines 42 shown in Figure 15(A), extending on 
two contiguous sides and meeting at the corner. This 
would allow us to ascertain precisely the location of 
the hit point and the orientation (i.e., the angle) of the 10 
specified pattern. In the extraction of the pattern, a 
specified distance can be obtained from the apex of 
the angle formed by guidelines 42. Then by extracting 
in the specified location, we can determine accurately 
the location of each cell. The subsequent character- 15 
istic count will also be accurate. This scheme also al- 
lows the subsequent fuzzy matching procedure to be 
performed rapidly and accurately. 

It would also be possible to provide guidelines 
42a in each corner of the grid, as shown in Figure 20 
15(B), or to surround the grid with guidelines 42b, as 
shown in Figure 15(C). In these cases, we would not 
be able to ascertain which edge is the top of the grid: 
however, we would be able to accurately determine 
the location of the specified pattern and from this to 25 
accurately obtain the location of each cell. These 
schemes, then would yield a result identical to that of 
this example. In the three examples given above, we 
printed only the dark marks 41a which had been col- 
ored. The light marks, those which are not colored, 30 
are not printed, nor are their (circular) outlines. How- 
ever, they may be printed if desired. 

It would be acceptable to create circular lines 43 
in the portions which are not to be colored, in addition 
to colored marks 41a, as shown in Figure 15(D). In 35 
this case, marks 41a and circles 43 together form the 
shape of the square. This entire segment can be ex- 
tracted. From the relationship between adjacent 
marks 41a and circles 43, we can obtain an accurate 
location for each cell (i.e., each region for which a 40 
characteristic count is to be extracted). When detec- 
tion device 1 lowers the resolution and gradates the 
image, setting the threshold value at an appropriate 
level will cause the portions occupied by the specified 
pattern to be entirely colored. This will allow the pat- 45 
tern to be detected and extracted with relative ease. 

We shall now discuss the fuzzy knowledge stored 
in the aforesaid memories 36 and 37. Let us assume 
that the specified pattern has nine marks (or cells), 
which we shall call C11 through C33, as shown in Fig- so 
ure 16. Let us also assume that Ceils C11, C22 and 
C33 are the dark marks. In this example, for each 
mark which indicates a bit datum, the density of the 
mark is expressed as a membership function which 
represents a characteristic count These membership 55 
functions are stored in MF memory 37. More specif- 
ically, C11, C22 and C33 have a high goodness of fit 
on the dark side, as snown in Figure 1 7; the other cells 



have a high goodness of fit on the light side. The 
width, shape and other characteristics of each mem- 
bership function are set so as to be able to cope with 
shifts of position. To give an example, the functions 
of cells C13 and C31, which would be far from the 
dark marks even if the pattern are shifted slightly, 
can be made steeper. 

For the example shown in Figure 16, the following 
rule would be stored in rule memory 36. 

If the characteristic counts for C 11 , C22 and C33 
are high and those for C12, C13, C21, C23, C31 and 
C32 are T times lower, then this is the specified pat- 
tern. 

In the actual processing performed by matching 
device 3, characteristic spaces (regions consisting of 
nine cells) are set up using pre-gradated image data. 
The number of characteristics is extracted by count- 
ing the actual number of colored (i.e., black) pixels in 
each characteristic space. In this way we obtain the 
pixel count for each characteristic space. Character- 
istic spaces which have colored marks will have col- 
ored pixels in the portions covered by those marks, 
which when counted will increase the characteristic 
count. (I.e., these spaces are darker.) 

Fuzzy inference unit 35 reads out the data stored 
in memories 36 and 37 and, using the extracted char- 
acteristic count as input, performs a fuzzy inference 
based on the rules and membership functions (shown 
in Figure 1 7) which were previously established. It de- 
termines and outputs the goodness of fit (the degree 
of resemblance) of the candidate with respect to the 
specified pattern. 

The goodness of fit obtained as a result of the in- 
ference is output to a P PC (a copier) by way of CPU 
30. When the goodness of fit exceeds a given thresh- 
old value, the copier will conclude that the document 
presented is a bank note or some other item which 
cannot be legally copied. It will proceed to execute the 
program it has for prevention of forgery (prohibiting 
the copying operation, outputting a totally black 
screen, and so on). As an alternative, CPU 30 can 
render a judgment as to whether the item presented 
is a prohibited document (i.e., whether it contains one 
of the specified patterns to be detected), and it can 
output the result (as, say, a stop signal) to the copier. 

With the example described above, the same 
specified pattern containing predetermined coded 
data (bit data) can be applied in a specified location 
on various kinds of documents which may not be cop- 
ied. Then, no matter what type of document it may be, 
the pattern will be detected. By applying the specified 
pattern to new varieties of bank notes, stock certifi- 
cates, and so on as they are issued, we can ensure 
that this image processing device will be widely effec- 
tive in detecting documents which may not be copied, 
and that it can be used semipermanently. Also, with 
this example only patterns which resemble the spe- 
cified pattern are extracted as candidates, and only 
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those patterns are matched against the specified 
pattern. Thus the specified pattern may be placed 
anywhere on the document which may not be copied. 

In this example, we discussed an example in 
which the same specified pattern is applied to various s 
documents. However, this invention is not limited to 
this idea. Rather, as shown in Figure 18, the bit data 
may be used purposefully. By varying the arrange- 
ment of colored marks 41a, we can have the pattern 
express different meanings. 10 

When several specified patterns are used, as in 
this case, inference unit 35 must obtain the goodness 
of fit of an extracted pattern with respect to each of 
specified patterns A through C. It may, for example, 
follow the flow pictured in Figure 19 to determine 15 
which of the specified patterns has been found. 

When several specified patterns are provided in 
this fashion, the various patterns can be placed in dif- 
ferent locations on a single document Then if all the 
patterns, or at least a given number of them, are found 20 
on a document, the copying operation can be prevent- 
ed. The use of multiple checks minimizes the prob- 
ability that the proscribed document will not be recog- 
nized. If one of the patterns has become hard to de- 
tect because it has become abraded or soiled through 25 
long use, the document can still be detected via the 
remaining patterns. When a single pattern is used, 
copying will be prevented if a pattern identical to one 
of the specified patterns happens to be found among 
the density data (in the characteristic count) from an 30 
ordinary document By requiring the matching of a 
number of specified patterns, we can reduce the 
chance that the copying of an ordinary document 
which should be allowed is prevented. 

The aforesaid coded data can be made to corre- 35 
spond to the type of bank note or stock certificate, or 
in the case of a bank note, to the denomination. Then 
when this system is operating and the copying proce- 
dure with respect to a prohibited document has been 
completed, the type of pattern detected and the fact 40 
that a stop command is issued can be stored, so that 
we can determine what type of documents are placed 
for copying in this device, or more specifically in the 
copier in which this device is installed. 

In the example discussed above, sampling and 45 
characteristic count extraction are performed on the 
entirety of each cell; however, the invention is not lim- 
ited to this design. It would also be possible, for ex- 
ample, to color all of a given cell as shown in Figure 
20, and sample only a part of it Sampling region 44a 50 
might be round and have a diameter smaller than the 
width of the cell, as shown in the drawing. This 
scheme can cope effectively with the shifts of posi- 
tion pictured in Figures 14 (A) and 14 (B). It allows the 
processing for extracting and removing the pattern to 55 
be simplified, since the corner of the cell can clearly 
be distinguished (i.e., the corner is highly visible be- 
cause the color extends all the way to it). 



In the example discussed above and its modified 
version, the mark in each cell is either colored or left 
uncolored to produce 1 bit of data. It would also be 
possible to provide a mark of intermediate density, 
which would produce multivalued data in a single cell. 
This would allow various types of data and classifica- 
tions to be distinguished using a small number of 
cells. It would also allow more complex marks to be 
created which would be more difficult to decode. 

Let us now consider some concrete schemes by 
which the density of the marks may be varied. Figure 
21 (A) shows different densities created by saturating 
the entire mark with color, using hatching to leave 
some portions white, and using progressively smaller 
shapes in the mark (in the example shown, three con- 
centric circles are used). Figure 21 (B) shows how the 
density may be reduced entirely in step fashion. The 
same effect could be achieved by varying the color 
which is used. 

Both the example discussed above and its modi- 
fied version use specified patterns which consist of 
square cells arranged in a 3 x 3 grid; however, the in- 
vention is not limited to this design. Any number of 
cells may be used. Figure 22 shows an arrangement 
of n x m cells (here n = m, but they could also have 
different values). As Figure 23 shows, it would also be 
possible to arrange cells 47 in concentric circles. 
Each cell would then be fan-shaped or the shape of 
a fan with its inner portion removed. Dark marks 48a 
and uncolored marks 48b would then be arranged in 
some appropriate fashion in these cells 47. Since the 
overall pattern will be a circle, the processing to ex- 
tract a hit mark in detection device 1 and extraction 
device 2 will use a search for a mark. 

When this scheme is used, the probability of a 
shape resembling the specified pattern appearing on 
an ordinary document is extremely small, so the 
chance of a recognition error is minimized. Here too, 
it would be possible to use multivalued densities in 
each cell, or to color entire cells 47. In this version the 
specified pattern may be represented in a variety of 
ways; as a circular frame delineating the contour of 
the pattern, which is partitioned into cells by linear 
segments L, as shown in Figure 23 (A); or only the cir- 
cular frame may be shown without lines to partition 
the cells, as in Figure 23(B); or without even the cir- 
cular frame, as shown in Figure 23(C). When, as in 
this case, there is a large number of cells, it is not nec- 
essary to use every cell to render a judgment. A char- 
acteristic count can be extracted from a portion of the 
cells and a determination can be made from that 
count as to whether this is the specified pattern. 

Figure 24 shows three other examples of speci- 
fied patterns which may be used with this invention. 
In Figure 24 (A), multiple patterns are provided. Area 

45, which occupies a portion of a specified mark con- 
sisting of n x m cells, is a singte unit; peripheral area 

46, consisting of 9 cells, specifies the type of docu- 
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ment. A commonly used design which does not con- 
sist of bit data is shown in Figures 24 (B) and (C). Here 
an ordinary mark (in this example, the letter A) is used 
instead. 

With this scheme, the goodness of fit is calculat- s 
ed for the pattern as a whole. If this value exceeds a 
given threshold value, we focus on area 46, which 
contains the bit data. The resolution of this area is in- 
creased and a characteristic count is extracted cell by 
cell. The patterns are then matched using fuzzy infer- 10 
ence, which allows them to be discriminated rapidly 
and accurately. Apattern like this (one which contains 
both a mark and bit data, for example, an "A" with bit 
data above it and to its right) has only a small prob- 
ability of occurring in an ordinary document (one 15 
whose reproduction is not prohibited), so the probabil- 
ity of mistaken identification is very small. 

A pattern which combines a mark with bit data 
can also take one of the forms shown in Figure 25. 
Cells 47 are formed by concentric circles. In these 20 
cells are marks 48a and 48b. In a specified location 
among these cells (here, in the center) is a mark 49. 

The shape of the cells and the overall pattern is 
not limited to those described in the example dis- 
cussed above or its modified versions. A combination 25 
of triangular cells could be used instead, or any ar- 
rangement in which at least a portion of the pattern 
contained coded data. 

Figures 26 and 27 show examples in which the 
aforesaid device is built into a copy machine. As 30 
shown in the drawing, the light emitted by lamp 52 is 
reflected by document 51, which is loaded on glass 
50. The reflected light passes through optical system 
53 and enters CCD 54, which is an image scanner. 
Here the image on the document is scanned. Lamp 35 
52, as well as the flat mirror and other components 
which comprise optical system 53, moves at a sped- 
fied speed as it scans the document. Line by line, 
CCD 54 scans the specified portion of document 51, 
and it transmits the image data (R/G/B) to signal 40 
processing unit 55. 

Signal processing unit 55, which is pictured in 
Figure 27, has an ordinary color processing circuit 56 
and device 57, which is the image processing device 
discussed above as a feature of this invention. The 45 
aforesaid image data is transmitted in parallel to color 
processing circuit 56 and to image processing device 
57. Color processing circuit 56 analyzes the colors 
into their various components, magenta (M), cyan 
(C), yellow (Y) and black (Bk), and outputs these data 50 
to printing device 58. The image is actually scanned 
four times. One of the aforesaid four components (M, 
C, Y, or Bk) resulting from the first scan is output to 
laser driver 59, which is positioned on the input side 
of printing device 58. This causes a laser beam to be 55 
projected onto a specified location on light-sensitive 
drum 60. When the fourth scan has been completed, 
the copying operation is performed on the copy pa- 



per, and the finished copy 61 is output. Since the ac- 
tual mechanism which executes the copying opera- 
tion is identical to that of existing copiers, we shall 
omit an explanation of its action. 

While the aforesaid signal processing is being 
carried out by color processing circuit 56, image proc- 
essing device 57 is performing its own processing in 
parallel to obtain the degree of resemblance between 
the pattern it finds in the image data it is scanning and 
the specified pattern. If document 51, which is still be- 
ing scanned, is a bank note or other document which 
may not be copied, the device will generate a control 
signal to halt the output of the aforesaid laser driver 
or transmit a control signal to color processing circuit 
56 to cause it to execute its various routines for sit- 
uations when copying is prohibited, such as making 
the entire screen go black. 

The judgment processing in this example is per- 
formed in real time during the first scan. This device, 
then, can be used in copiers which employ a single- 
scan method. Since only a portion, and not the entire 
face, of a bill is detected and matched against a spe- 
cified pattern, none of the requisite memories need 
have a large capacity. 

In the example discussed above, the device is in- 
stalled in a copy machine; however, this invention is 
not limited to this application, but could be applied as 
well in a color scanner, a color printer, a fax machine, 
a device to transmit communications, or a variety of 
other devices. 

The image processing device of this invention 
and the copy machine in which it is used have a lower 
cost than their predecessors because they require 
only a small-capacity memory. This improvement is 
a result of the fact that the specified patterns which 
are detected constitute only a portion of the entire im- 
age. Thus these patterns can be detected regardless 
of the size (i.e., the exterior dimensions) of the docu- 
ment which may not be copied (or printed). Detection 
is based on whether or not these specified patterns 
can be found. All processing from detection of possi- 
ble specified patterns in the first scan to determining 
their degree of resemblance to a reference pattern 
can be performed in real time. The specified pattern 
will not be missed even in a single-scan copy ma- 
chine, and the recognition rate will be high. 

A given pattern to be detected will have at least 
a portion which contains coded data which were de- 
termined ahead of time. By printing or otherwise ap- 
plying this specified pattern somewhere on the bank 
note, stock certificate, or other item which may not be 
copied, we can insure that a proscribed document will 
be detected accurately regardless of what type of 
document it is or what its shape or appearance may 
be. There is virtually no possibility that an ordinary 
document would have on it a shape identical to a spe- 
cified pattern in at least a portion of which there are 
coded data. This minimizes the probability of false 
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positives. Since the object to be detected (i.e., the 
pattern) is already determined, the image processing 
device can be given knowledge of this pattern. Then 
should any new document be issued which may not 
be copied, the same pattern can simply be imprinted 
on it, and it too will be detected. Because the speci- 
fied pattern is the only item which needs to be detect- 
ed, the invention can be applied to a wide variety of 
documents which may not be copied, and the proc- 
essing can be performed at high speed. 

If for every search the received image is gradated 
and potential matches are compared to the gradated 
image (or pattern), the image data with the specified 
pattern to be detected can be extracted quickly and 
accurately. 

If the aforesaid specified pattern comprises a 
portion which does not consist of coded information 
and/or a portion which does, and one of a number of 
different patterns is incorporated in it, a complex 
shape or contour may be used to minimize the prob- 
ability of a similar pattern being found on an ordinary 
document. Furthermore, because the types of 
shapes which can be represented are quite limited, 
the device requires only a small amount of fuzzy 
knowledge, and processing can be performed more 
rapidly. 

If the density data for each cell are multivalued, 
various types of documents could be distinguished 
using a small number of cells. The device could then 
be used to determine the type of document loaded 
into the copier. If a fictitious mark whose diameter is 
less than the width of a cell is established in a speci- 
fied location within a given cell, and the density of this 
mark is varied, or the sampling region used to obtain 
the density of the aforesaid cell is the region in a spe- 
cified location in that cell which is within a circle 
whose diameter is less than the width of the cell, the 
device will be able to cope effectively with shifts of 
position. 

By installing this image processing device in a 
copy machine, we can absolutely prevent the output 
of copies of documents whose reproduction is illegal, 
such as bank notes and negotiable securities. This 
entails refusing to copy the prohibited document 
while allowing other documents to be copied and out- 
put. 

Second Embodiment 

Herein we shall discuss an ideal second embodi- 
ment of the image processing scheme and device of 
this invention as well as a copy machine and other de- 
vices employing this scheme and device, with refer- 
ence to the appended drawings. As in the first em- 
bodiment. Figure 1 shows our first example of this 
second embodiment. As is shown, the image data 
read by the image sensor in the copy machine are in- 
put into pattern detection device 1, which detects the 



positioning marks. As these image data are scanned 
by an image sensor such as a CCD, they are trans- 
mitted successively, region by region, to device 1. 
The actual data which are sent are full color data for 

5 each of the red (R), green (G) and blue (B) compo- 
nents, with resolution of 400DPI. 

Using masks of specified sizes, detection device 
1 checks the area of this image via pattern matching 
for positioning marks. It specifies a reference position 

10 (the center of gravity for a triangle, the apex of a cor- 
ner for a rectangle, and so on) for any pattern desig- 
nated by these marks, which is a candidate for a 
match, and it transmits the relevant data to device 2, 
the device to extract a specified pattern. 

15 Based on the position (i.e., reference position) 

data which it receives, extraction device 2, with the 
help of the control CPU, extracts a specified portion 
of the image which is read. Device 2 creates from 
these extracted data a pattern to be compared to a 

20 reference in the pattern matching process to be per- 
formed later. It sends this pattern to device 3, which 
performs the next stage of the processing, the com- 
parison of two patterns. 

Matching device 3 uses fuzzy inference to obtain 

25 the goodness of fit of the extracted pattern with re- 
spect to the reference pattern (i.e., the specified pat- 
tern), and it determines whether the extracted pattern 
is, in fact, the specified pattern. If it is, the device out- 
puts a specified control signal. 

30 In this example of a data processing scheme ac- 

cording to this invention, then, ordinary data read by 
a copy machine are used. On the first scan, the pos- 
itioning marks are detected by device 1. A pattern 
which is a potential match (i.e., the dimensions of a 

35 region to be extracted) is calculated based on the lo- 
cation of these marks, and this pattern is extracted by 
device 2 on the second scan. The matching device 
performs the specified processing sequentially on 
the pattern in the extracted region. This pattern is 

40 then matched against a reference using fuzzy infer- 
ence, and the device determines whether this is the 
specified pattern. 

The actual pattern matching performed by de- 
vice 3 uses a detailed image which is relatively close 

45 to the original. Before this takes place, however, the 
number of patterns to be matched is narrowed down 
by the search for positioning marks. The search for 
these small marks is si mpie and fast and it allows the 
fuzzy inference processing to be performed on only 

50 a few candidates. The result is that the overall proc- 
essing can be performed at high speed. While each 
process is being performed, the processing per- 
formed by the other devices, for example, the reading 
of the image or the search for positioning marks, is 

55 also in progress. Thus there is no need to halt the 
processing while a determination is being made, and 
all processing can be performed rapidly in real time. 
We shall next discuss an actual example of the 
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image processing scheme described above. We be- 
gin by discussing the positioning marks used in this 
example. As is shown in Figure 28, positioning marks 
4, which are surrounded by squares consisting of 
broken lines, here consist of triangles. At least three 5 
(and in this case four) marks 4 are arranged in a given 
relationship. These four marks are placed on the apic- 
es of an asymmetrical quadrilateral (some shape 
other than a square). 

In this way, when marks 4, arranged in their given 10 
relationship, are detected, the coordinates of their lo- 
cations can be used to determine unmistakably the 
orientation and position of the document loaded in the 
copier as well as the location of specified pattern 6 
with respect to marks 4. Positioning marks 4 can be 15 
detected in the recognition processing of the first 
scan. Based on these marks, the location of a pattern 
which is a candidate for matching can be specified ac- 
curately and swiftly, and the processing can proceed 
at high speed. 20 

If the coordinate position of each mark 4 is spe- 
cified, the marks can be made smaller (in comparison 
to guidelines). Although this will make them more dif- 
ficult for an uninformed person to discern, the rate of 
recognition will remain high. Making the marks small- 25 
er will also allow a smaller window to be used for their 
detection, and it will allow the detection circuit to be 
simplified and constructed of cheaper components. 
Smaller marks will also contribute to processing 
speed. 30 

In this example, numerous dummy marks 5 which 
resemble positioning marks 4 are arranged around 
their periphery. To be more specific, these dummy 
marks are triangular, as are the positioning marks; but 
the dummy triangles have shorter sides. Since there 35 
are so many similar marks, a person glancing at them 
would be unlikely to notice that some of them serve 
as positioning marks. It would also be difficult to de- 
termine which of these marks constitute the position- 
ing marks. This makes it more difficult to tamper with 40 
the bill; and if a document which may not be copied 
is being processed, it will be detected reliably. In the 
example shown in the drawing, dummy marks 5 have 
all been made the same shape for the sake of conve- 
nience (they are slightly smaller triangles than posi- 45 
tioning marks 4); however, if a number of different si- 
zes of dummy marks 5 are arranged appropriately, it 
will be even more difficult to distinguish the real pos- 
itioning marks 4. 

When this arrangement is used, in the examples 50 
shown, pattern 6, the "A" surrounded by positioning 
marks 4, will be extracted and removed as a candi- 
date for matching, while pattern 7, the "B" without 
positioning marks, will not In this example the posi- 
tioning marks 4 are enclosed in squares drawn in 55 
broken lines for the sake of the explanation; however, 
these squares would not be printed on an actual docu- 
ment. 

18 



The specified pattern need not be limited to a let- 
ter or symbol, as shown in Figure 28. It could, for ex- 
ample, as shown in Figure 29, consist of a set of ele- 
ments 6'a arranged in a specific relationship in region 
6', which is surrounded by positioning marks 4. 

If, as shown in the drawing, elements 6'a are giv- 
en the same shape as dummy marks 5, it will be very 
difficult to determine where the positioning marks 
are, which marks constitute the specified pattern, 
and which marks are dummies which are unrelated to 
finding the pattern. This will minimize the probability 
of tampering. 

Pattern elements 6'a need not be identical to 
dummy marks 5. They may be similar to marks 5, or 
they may be identical or similar to positioning marks 
4. It would also be possible for the pattern elements 
to have a different shape u nrelated to any of the other 
marks. 

As is shown in Figure 30, dummy marks 4'a 
through 4 f c may be used which are identical to posi- 
tioning marks 4a through 4d. In this example, when 
each positioning mark is found to coincide with its pre- 
dicted coordinates and the positions of the marks rel- 
ative to each other also correspond to those predict- 
ed, the marks are recognized as the real positioning 
marks, and the region they designate is extracted. If 
one of these marks is missing (in the example shown, 
the dummy mark corresponding to 4d), the marks will 
not be recognized as positioning marks, and pattern 
7, the letter "B M , will not be extracted. When dummy 
marks are used which are identical to the positioning 
marks, a person who attempts to make improper use 
of the document will find it extremely difficult to de- 
termine correctly the location of the positioning marks 
and/or the specified pattern. 

In all the examples shown thus far, the specified 
patterns 6 and 6' are placed in the regions surround- 
ed by positioning marks 4 and 4a through d; however, 
the invention is not limited to this condition. The spe- 
cified pattern may be placed somewhere outside the 
marks, as long as it is in a given relationship relative 
to those marks. 

If we use Figure 30 as an example, we could call 
the letter "B B the specified pattern and define it (i.e., 
define the region containing the item to be extracted) 
as being a specified distance away from the longest 
side (4d-4c) of the trapezoid formed by marks 4a 
through 4d. Because the quadrilateral defined by 
marks 4a through d is asymmetrical, as discussed 
above, the region to be extracted and removed can be 
determined unmistakably, and it can be specified 
simply and precisely. 

We shall now discuss in detail the actual config- 
uration of each component of the device shown in Fig- 
ure 1 , our first ideal example of the image processing 
device of this invention, in which the aforesaid 
scheme may be implemented. An example of pattern 
detection device 1 is shown as pattern detection de- 



BNSDOC1D <EP 0664642A2_I_> 



33 



EP 0 664 642 A2 



34 



vice 1" in Figure 31 . The image data received (graded 
data (dark- light data) corresponding to RGB data) are 
converted to binary data at a given threshold by bina- 
ry processing unit 23. The converted binary data are 
stored temporarily in multiple line memory 1 2. By set- 5 
ting an appropriate threshold for the conversion to bi- 
nary, we can produce an image in which, for example, 
pixels occupied by positioning marks will appear 
black while those surrounding them (the background) 
are totally white. 10 

The output of the said binary processing device 
23 is sent to search unit 26 by way of line memory 12. 
Search unit 26 uses a mask of dimensions n x m 
(where n and m may have the same value or different 
values) to search the image data which it has received 15 
(after it is converted to binary data) for the specified 
pattern (here, the positioning marks). Specific details 
are given below. 

Search unit 26, shown in Figure 31, is composed 
of flip-flop group 26a; circuit to output coordinates 20 
26b, which receives the output from flip-flop group 
26a, obtains the coordinates of the hit points (in this 
example, the center coordinates of each cell (or pixel) 
of the mask) which indicate the positions of the spe- 
cified pattern (here the positioning marks); decoder 25 
circuit 26c, which checks the mask for positioning 
marks; and determination unit 26d, which receives 
the output from decoder circuit 26c and, in the event 
that the positioning marks have been detected, ob- 
tains and outputs their center coordinates. 30 

The actual arrangement of flip-flop group 26a is 
shown in Figure 32. In this example, for the sake of 
convenience, the area being processed will consist of 
3 pixels of scanning x 3 pixels of feed, or 9 (3 x 3) flip- 
flops FF. The three pixels W10, W1 1 and W12, which 35 
are in line in the feed orientation, are taken from the 
image data stored in line memory 12, synchronized 
by means of the shift clock SCLK, and input to the first 
column of flip-flops, F00, F1 0 and F20. New pixel data 
are input to W1 0, W1 1 and W1 2 sequentially accord- 40 
ing to timing provided by the shift clock, and the data 
are transmitted to the next column of flip-flops. The 
pixel data which are input for the binary image, Q00, 
Q01, Q02, Q10, Q11, Q12, Q20, Q21 and Q22, are 
output through the output terminal Q on each flip-flop. 45 
For a black pixel, the output of the flip-flop will be "1\ 

When all scanning data up to the end of one line 
have been input, the device shifts down one line in 
the direction of feed and inputs data from the begin- 
ning of that line. In this way, the 3x3 pixel area (the so 
mask) which is the domain of the search is moved 
over the surface of the document. 

Each of the aforesaid outputs, Q00, Q01, Q02, 
Q10, Q11, Q12, Q20, Q21 and Q22, is input into de- 
coder circuit 26c. Decoder circuit 26c is pictured in 55 
Figure 33. If the pattern of 3 x 3 pixels which is output 
by flip-flop group 26a is a given pattern (such that the 
central pixel is black and the eight surrounding pixels 

19 



white), the output of the corresponding AND element 
will go high. If the hit output from decoder circuit 26c 
goes high, we know that there is a positioning mark 
centered around flip-flop FF11. 

Circuit 26b is connected to the addresses in mul- 
tiple line memory 12. It outputs to flip-flop group 26a 
the addresses of images which must be input, and it 
outputs to determination unit 26d the coordinates 
(XY) of the pixels output by FF1 1 , the central flip-flop 
in group 26a. The output of decoder circuit 26c and 
the XY coordinates of the positioning mark at that mo- 
ment are input to determination unit 26d. If a hit output 
goes high, unit 26d stores its XY coordinates in mem- 
ory 27, the memory for coordinates of positioning 
marks. In this example, we used nine flip-flops to 
search for a positioning mark using a mask of 3 x 3 
pixels, and the states of the decoder circuit are those 
shown in the diagram. However, these conditions are 
chosen arbitrarily for the convenience of the explan- 
ation. The number of flip-flops used and the condi- 
tions for the decoder must be selected according to 
the shape of the pattern one wishes to detect and 
which portion of it one chooses to focus on. 

As is shown is Figures 28 through 30, in this ex- 
ample we use dummy marks 5, which are slightly dif- 
ferent in size from positioning marks 4. It is thus nec- 
essary that we be able to detect marks 4 without de- 
tecting dummy marks 5. The actual dot patterns form- 
ing marks 4 and 5 are pictured in Figures 34 (A) and 
(B) (Each cell S corresponds to one pixel). This being 
the case, flipflop group 26a would not consist of the 
3x3 pixels shown in Figure 32, but of 10 x 10 pixels 
corresponding to the cells S shown in Figure 34. The 
decoder circuit for this purpose would have a one hun- 
dred-input AND element (or an equivalent combina- 
tion of AND elements). The input terminals corre- 
sponding to the black pixels in Figure 34 (A) would be 
non-inverting, but those corresponding to the white 
pixels would be inverting inputs. Then the hit output 
would go high only when the pattern shown in Figure 
34 (A) is found, and low in every other case (including 
the pattern in 8 (B)). When the hit goes high, just as 
discussed earlier, the XY coordinates are output and 
stored. 

When the positioning marks to be detected are 
triangles, as in this example, detection may depend 
on differences in angles. For example, decoder cir- 
cuits 26c corresponding to a number of patterns with 
different angles (the portions for which input is invert- 
ed are composed of AND elements varied appro- 
priately to correspond to the pattern), may be con- 
nected in parallel to the output of flip-flop group 26a. 
When one of the decoder circuits detects a given an- 
gle, we conclude that a positioning mark has been 
found. 

If a positioning mark consists of a single pixel, the 
3x3 pixel mask (or window) discussed above can be 
used just as it is. In this case there is a possibility that 
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a printing irregularity might result in a spot the size of 
a pixel being printed somewhere that it does not be- 
long. Even then, the scheme still entails the detection 
of a number of positioning marks in a given relation- 
ship to each other, so a spurious spot could imme- 5 
d iately be distinguished before the region to extract is 
calculated. In practice, then, such a spot would pose 
no problems. 

If we consider the example in Figure 34, we can 
use a processing unit of a size sufficient to contain a 10 
single positioning mark (for example, 8x8 pixels) . If 
we average an 8 x 8 pixel square and convert it to a 
binary value, the same square will be above the bina- 
ry threshold value and so will go to black in Figure 
34(A), while it will be below the threshold and so will 15 
go to white in (8). If these units, after they are aver- 
aged and converted to binary values, are input to the 
flip-flops shown in Figure 32, the circuits shown in 
Figures 32 and 33 can be employed without modifi- 
cation. 20 

This scheme would require an averaging unit 
either before or after binary processing unit 23 in Fig- 
ure 31 to obtain the average value of a given number 
of pixels. The threshold value used to detect position- 
ing marks as discussed above should ideally include 25 
both an upper and a lower limit When the binary con- 
version is performed using a window comparator, the 
entire inside of the shape formed by the black pixels 
in Figure 34 (A), for example, will go to black, causing 
the pattern to exceed the upper limit and so be ruled 30 
out. 

Figure 35, showing device 2" is a block diagram 
of device 2, the device to extract the specified pat- 
tern. As can be seen in the drawing, the position co- 
ordinates of the positioning marks output by detection 35 
device 1" on the first scan are stored in coordinate 
memory 27. The data stored in memory 27 are sent 
to control CPU 38, which determines whether each 
set of position coordinates is in a given relationship to 
the others. If this is the case, i.e., if the real position- 40 
ing marks have been found, it obtains (the coordin- 
ates of) a region to be extracted, based on these pos- 
ition coordinates. 

More specifically, as can be seen in Figure 36, we 
use a reference line L which is horizontal with respect 45 
to the glass of the copy machine, to obtain from the 
coordinates of the four positioning marks (Xi, yi), (x 2 , 
Y2>. (x 3 , V3) and (x*, y 4 ) the slope 0T of specified pat- 
tern 6, which is found in a given location relative to 
these marks, and the reference coordinates (x 5 , y 5 ) of 50 
the region to be extracted (here shown by dotted 
lines). The relevant processing is performed by con- 
trol CPU 38, and the data concerning the extraction 
of the specified pattern are stored temporarily. Since 
the size of the region to be extracted (as well as its 55 
shape) is already known, determining the said refer- 
ence coordinates and slope (which are temporarily 
stored) allows us to specify quite precisely the region 



to be extracted. 

Device 2*\ the device to extract the specified pat- 
tern, is sent the original, non-binary RGB data (the 
dark-light data) by detection device V*. These image 
data are converted to binary data by conversion unit 
39, and the resulting binary data are stored in mem- 
ory 65. The threshold for the conversion to binary is 
not the same one used in the aforesaid binary proc- 
essing unit 23 of detection device 1 ". This allows the 
positioning marks and the specified pattern, which is 
composed of various common colors, to be converted 
to binary values using optimal criteria for each. 

In response to a control signal from CPU 38, the 
region designated by the said reference coordinates 
(*s, ys) and the angle 0T of the slope which are ob- 
tained (and shown by the dotted line in the figure) is 
extracted from the binary image data stored in mem- 
ory 65. CPU 38 performs the specified characteristic 
count and sends it to pattern matching device 3". 
Some of the functions of control CPU 38 are.con- 
tained in extraction device 2". Aflowchart of the func- 
tions of participating components is shown in Figure 
37. 

We shall now discuss pattern matching device 
3". Based on the data stored in program ROM 38a, 
control CPU 38 specifies which of the image data 
stored in memory 65 must be read out, and it decides 
which region of the image data should be matched 
against the reference pattern. The relevant image 
data in this region are transmitted from memory 65 to 
inference unit 31 in response to a control signal from 
CPU 38. Inferences are processed using the fuzzy 
knowledge, which includes the rules and membership 
functions stored in memories 32 and 33. A judgment 
is then rendered as to the degree of resemblance be- 
tween the received image data and the previously es- 
tablished specified pattern to be detected (See Fig- 
ure 12). 

A specific example is given in Figure 38. The 
number of dark pixels in each characteristic space in 
the image data which is extracted (in this example 
there are four) is counted to obtain a characteristic 
count. In this way we obtain a count of 24 pixels in 
space R 1( 26 in space R 2 , 30 in space R 3 and 29 in 
space R 4 . This count is performed by control CPU 38. 

A fuzzy inference is performed using the charac- 
teristic count which was obtained, according to previ- 
ously established rules and membership functions 
(see Figure 39). The goodness of fit (i.e., the degree 
of resemblance) of the extracted image with respect 
to a reference pattern is determined and outputted. In 
this example, the goodness of fit is 0.6. Since this is 
greater than 0.5, which is the reference value for judg- 
ments, it is concluded that the image is identical to the 
reference pattern. 

In this example the image is divided into spaces 
for the characteristic count by means of a lattice; how- 
ever, the image could also be divided into concentric 
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circles, rectangles, or any other desired shape. The 
characteristic count need not be limited to the number 
of dark pixels. It would be equally acceptable to count 
the number of connected or disconnected pixels, pix- 
els arranged in a certain orientation, or some other 5 
feature. 

As indicated above, the goodness of fit is ob- 
tained as a result of an inference based on a number 
of items of fuzzy knowledge. This goodness of fit is 
output to a PPC (the copy machine) by way of control 10 
CPU 38. If the goodness of fit exceeds a given thresh- 
old value, the copy machine will conclude that it is 
something which may not be copied, such as a bank 
note, and it will execute its specified processing to 
prevent counterfeiting (prohibiting the copying oper- 15 
ation, outputting a totally black screen, etc.). The de- 
termination of whether this item is a prohibited docu- 
ment such as a bank note (i.e., whether it matches the 
specified pattern) could also be made by CPU 38, 
which would then output the result (a stop signal or 20 
the like). 

As in the first embodiment, figures 26 and 27 
show an example in which the aforesaid device 57 is 
built into a copy machine. In this example discussed 
above, the positioning marks are detected on the first 25 
scan and the pattern is extracted and its degree of re- 
semblance to the reference pattern calculated on the 
second scan. However, there is no inherent reason 
why the order of processing could not be changed, as 
there is no need to perform this processing on the first 30 
and second scan. Each process can be executed on 
whatever scan the user finds most convenient. If 
there are many kinds of documents which must be de- 
tected, a given specified pattern could be detected on 
the first and second scans and a different specified 35 
pattern could be detected on the third and fourth 
scans. 

Similarly, the specified pattern need not be de- 
tected on the second scan. Particularly when the pos- 
itioning marks are quite close to the specified pattern, 40 
they could be stored in a buffer memory (which would 
allow a smaller memory to be used). Then after the 
marks are detected on the first scan, the location of 
the pattern could be determined and the extraction 
processing executed. 45 

In the example we have been discussing, both 
the positioning marks and the dummy marks are tri- 
angles which can be distinguished from each other by 
the fact that their sides are not of equal length. How- 
ever, the invention is not limited to this case only. A so 
variety of shapes would be equally acceptable. To 
give some examples, we could also use for the real 
marks and the dummies a combination of parallelo- 
grams with different slopes; isosceles triangles with 
different base angles; scalene triangles of the same 55 
height but having different slopes; or various poly- 
gons with different angles. In addition to varying the 
angles of the figures in this way, it would also be pos- 
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sible, as is done in the example, to vary the dimen- 
sions of the figures in an appropriate manner. 

Another scheme to produce marks and dummies 
would be to use a number of intersecting wavy lines 
M of a given thickness to form different shapes (See 
Figure 40). The positioning marks could then be dis- 
tinguished by the differences in the shapes of the in- 
tersections. As can be seen in the blow-ups in Figure 
40(B) and (C), differences in the shape of the line 
cause the shape of the intersection (shown by hatch- 
ing) to vary. We could, then, use these intersecting 
lines as positioning marks and detect an intersection 
of a given shape. With such a scheme, an uninformed 
person would be very likely to see the lines as simply 
a background pattern, which would optimally mini- 
mize the probability of tampering. 

Dummy marks need not be used at all. To identify 
unmistakably the location of the specified pattern, 
the positioning marks should ideally be placed on the 
apices of an isosceles triangle or an asymmetrical 
polygon (which may be a triangle or a rectangle). 
However, they could also be placed on the apices of, 
for example, an equilateral triangle, a square, or 
some other regular polygon. In this case, we might 
choose as a specified pattern for the center of a reg- 
ular polygon a symmetrical shape composed of lines 
lacking directionality in either the vertical or the hori- 
zontal plane. This would ensure that the pattern could 
be located unmistakably on the first scan. 

In all of the examples discussed thus far, four pos- 
itioning marks have been detected; however, three or 
five or more marks could also be used at the user's 
discretion. If the shape of the marks is given a distin- 
guishing feature (say, directionality), it would be pos- 
sible to use only one or two marks. Although in these 
examples the positioning marks are all identical, the 
invention is not limited to this case. It would be equally 
possible to select positioning marks which are differ- 
ent from or similar to each other. In this case, the de- 
coder circuits corresponding to the various marks 
would be connected in parallel to the outputs of the 
flip-flop groups. Then when a mark is detected, its rel- 
ative position vis-a-vis the other marks can by 
checked (the second step of the processing in Figure 
37) before the control CPU determines the coordin- 
ates for extraction. Not only the spatial relationship of 
each mark with the others, but also the relationship 
between type of mark and position can be checked. 

Instead of varying their shapes, the real and dum- 
my marks may also be distinguished from each other 
by varying their respective colors. In this case, we 
might, for example, use the binary circuit shown in 
Figure 41 for binary processing unit 23 in detection 
device 1". In this circuit, threshold values for upper 
and lower limits are set for each of R, G and B by val- 
ue setting units 23a and 23b. The RGB data which are 
received are input to window comparators 23c, which 
compare them with each threshold value. In this way 
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pixels of a given density (or tone) can be extracted. If 
and only if a pixel of the desired color is obtained, the 
output of AND element 23d, which receives the last 
three inputs, will be n 1 w . 

As we mentioned earlier, the specified pattern 
could be a portion of the document which has a high 
characteristic count (in this case, too, the pattern can 
be located accurately and simply by detecting posi- 
tioning marks); or it could be added separately to the 
document, just as the positioning marks are. 

When a characteristic image is selected as the 
specified pattern, the image processing device must 
be given knowledge concerning this pattern. Then 
should any new document be issued which may not 
be copied, the same pattern can simply be imprinted 
on it, and it too will be detected. Because the speci- 
fied pattern is the only item which needs to be detect- 
ed, the invention can be applied to a wide variety of 
documents which may not be copied, and the proc- 
essing can be performed at high speed. 

In the example discussed above, the judgment 
processing performed by pattern matching device 
3" consisted of obtaining the degree of resemblance 
of an extracted pattern to a previously established ref- 
erence pattern (i.e., the specified pattern). Other 
possible schemes include the one pictured in Figure 
42. Specified pattern 6" is composed of binary data 
expressed as black or white pixels in the extracted re- 
gion determined by positioning marks 4. The pattern 
matching device obtains a value of 1 or 0 from the 
density of each cell constituting pattern 6". They are 
then decoded as a string of Is and 0s in a specified 
order (in the example shown, the string is 
"0100000010..."). It can then be determined whether 
the decoded values correspond to the data represent- 
ing the reference code for the specified pattern, and, 
by the same token, whether this is in fact the specified 
pattern. 

Figure 43 shows second ideal example of the im- 
age processing device of this invention. This example 
differs from the previous one in that the specified pat- 
tern in the image is detected directly from the image 
data which are received (without determining coordin- 
ate values by detecting positioning marks). 

In this example, as shown in Figure 44, specified 
pattern 8 consists of a number of marks 8a and 8b, 
which are triangles of identical shape, arranged in a 
specific relationship. More specifically, a given num- 
ber of marks 8 (in this case, ten) are arranged in a 
shape (here the outline of a face in profile) with a sin- 
gle mark 8a in its center. Since in this example any 
pattern with the same number of marks 8 on its per- 
iphery constitutes the specified pattern, the actual 
placement of each mark 8 on the outline is arbitrary. 
Thus the placement shown is not the only one possi- 
ble. As long as there are ten marks on the outline of 
the shape, it wilt be identified as the specified pat- 
tern. 



The marks 8a and 8b constituting the specified 
pattern 8 used in this example are identical to the pos- 
itioning marks 4 in the first example discussed earlier, 
and their shapes have the same characteristics. 

5 One effect of the marks 4 discussed with respect 

to the first example is that because their coordinate 
positions can be specified, the marks themselves can 
be made quite small. This makes it harder for an un- 
informed person to discern, and there is no drop in the 

10 recognition rate. Making the marks smaller will also 
allow a smaller window to be used for their detection, 
and it will allow the detection circuit to be simplified 
and constructed of cheaper components. Smaller 
marks will also contribute to processing speed. Marks 

15 8a and 8b in this second example achieve the same 
effects. 

In the device pictured in Figure 43, device 70 de- 
tects the said marks 8a and 8b among the image data 
which it receives and obtains their position coordin- 

20 ates. The coordinates of each mark which is detected 
are sent to device 80, which performs the next stage 
of the process, detecting the specified pattern. From 
the locations of marks 8a and 8b, device 80 determi- 
nes whether the marks are in a given relationship with 

25 each other, like that shown in Figure 44, and it outputs 
the goodness of fit 

In the image processing scheme according to this 
invention which is implemented using this device, the 
i mage of the document is read on th e first of a number 

30 of scans and sent to detection device 70. This device 
makes a search for the aforesaid marks 8a and 8b. 
When it finds a mark, it obtains and stores the position 
coordinates of that mark. 

When at least the data corresponding to the wid- 

35 est part of the outline of the shape have been read 
(the judgment processing (i.e., the processing to de- 
tect the specified pattern) can of course also be be- 
gun when all the image data have been read), detec- 
tion device 80 is operated. From the position data for 

40 the marks which have been detected, this device 
checks whether the marks are in a given relationship 
in the image which is read, and in this way detects the 
specified pattern. 

More specifically, a window of a specific size (N 

45 x N) is established. Using the position coordinates of 
the marks, each mass which is a part of the window 
is checked for a mark. Masses on the outline of the 
given shape are checked for a given number of marks 
(in Figure 44, ten). If the requisite number is found, a 

so determination is made that this is the specified pat- 
tern. 

In view of the possibility of printing misregisters, 
smudges, or tampering, it is advisable to establish a 
fixed margin for the determination. That is, one would 
55 establish a zone of a fixed width on either side of the 
outline of the shape. Any mark found in this zone 
would be recognized as a component of the specified 
pattern. A given weighting can be used for the marks, 
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so that a lower weight is assigned as the distance 
from the outline increases. This will increase the ac- 
curacy of detection by minimizing the number of 
marks overlooked or detected by mistake. It is also 
advisable to provide a margin with respect to the 5 
aforesaid number of marks (here, ten). Then even if 
a mark is not recognized because it has faded over 
time or a part of it is missing due to a detection error, 
the pattern will still be detected correctly. 

In this example, then, the marks constituting the 10 
specified pattern on the document are detected and 
a determination is made, on the basis of the position 
coordinates of the marks, as to whether this is in fact 
the specified pattern. This being the case, all the data 
needed for a determination can be acquired on a sin- 15 
gle scan. The only check that is performed is the one 
to determine whether the position coordinates of the 
marks are in a given relationship (at a given distance) 
with respect to each other, so the processing can be 
performed at high speed. 20 

It is not necessary that a window, as described 
above, be used to detect the specified pattern. If the 
processing are executed via software, we could as- 
sume that a given mark is central mark 8a, then detect 
all the marks at a given distance from that mark (with- 25 
in a fixed margin, if desired) to obtain the applicable 
number of marks. If this number falls within a fixed 
range, we determine that the specified pattern has 
been found. 

. We shall now explain the actual configuration of 30 
devices 70 and 80 in the image processing device in 
Figure 43, the second ideal example in which the said 
image processing scheme may be implemented. We 
shall first discuss device 70, the device to detect the 
marks. The shape and other features of the marks 8a 35 
and 8b used in this example are identical to those of 
marks 4 in the example. Device 70 can thus be con- 
figured much as device 1, the device to detect the 
positioning marks in the example. 

As can be seen in Figure 45, the image data 40 
which are received (i.e., tonal or light-dark data cor- 
responding to RGB data) are converted into binary 
data at a given threshold value by binary processing 
unit 71. The newly converted binary image is stored 
temporarily in multiple line memory 72. By setting an 45 
appropriate threshold for the conversion to binary, we 
can produce an image in which, for example, a pixel 
where a mark 8a or 8b is found is appears black while 
the surrounding area (the background) is totally white. 

The output of the said binary processing unit 71 50 
is sent to search unit 73 by way of line memory 72. 
Search unit 73 uses a mask of dimensions n x m 
(where n and m may have the same value or different 
values) to search the image data which it has received 
(after it is converted to binary data) for marks 8a and 55 
8b, small circles of a given diameter. A detailed ex- 
planation is given below. 

Search unit 73, shown in Figure 45, is composed 



of flip-flop group 73a; circuit 73b, which receives the 
output from flip-flop group 73a, obtains the coordin- 
ates of the hit points (representative points in each 
cell (or pixel) constituting the mask, in this example, 
the center coordinates) which indicate the positions of 
marks 8a and 8b; and decoder circuit 73c, which 
checks the mask for marks 8a and 8b. The outputs of 
coordinate detection circuit 73b and decoder circuit 
73c are connected to memory 81, the memory to 
store the locations of marks, in detection device 80. 
When the output hit in decoder 73c goes high (indicat- 
ing the presence of a mark), the (X, Y) coordinates 
output by circuit 73b are stored in location memory 
81. 

The basic configuration of the said flip-flop group 
73a, as shown in Figure 32, is an area of 3 pixels of 
scanning x 3 pixels of feed, or 9 (3 x 3) flip-flops FF. 
In practice, the size of marks 8a and 8b and the num- 
ber of marks used are selected as appropriate. If 
marks 8a and 8b are like that shown in Figure 34 (A), 
it would be appropriate to use a 10 x 10 flip-flip group. 
The decoder circuit corresponding to these flip-flops 
could be composed of a one hundred-input AND ele- 
ment (or an equivalent combination of AND ele- 
ments). If the marks are of the size shown in Figure 
34(B), an 8 x 8 flip-flop group would be preferable. A 
suitable decoder circuit for these flip-flops would be 
composed of a 64-input AND element (or combination 
of AND elements). 

Image data stored in line memory 72 are supplied 
sequentially at a specified timing to a given flip-flop 
in flip-flop group 73. Since the relevant operation is 
identical to that of flip-flop group 26a, we shall omit 
further discussion. 

The output of every flip-flop in the flip-flop group 
is input to decoder circuit 73c. Decoder circuit 73c is 
based on the AND element shown in Figure 33. The 
circuit has as many inputs as there are flip-flops. With 
reference to Figure 34, the pixels which are to become 
black are input just as they are; those that are to be- 
come white have inverted input Once this is done, the 
output of an AND element will go high only when a cir- 
cle of a given radius r is found. If an output hit of de- 
coder circuit 73c goes high, this indicates that marks 
8a and 8b, which constitute the specified pattern, 
have been found centered around the central flip- 
flop. To allow for misregisters, printing irregularities 
and blurring resulting from long use, it is advisable 
that the output of each pixel not be connected directly 
to the input of an AND element, but rather, several 
neighboring pixels (two works well) be connected to 
the input of a single AND element. With this scheme, 
marks 8a and 8b will always be detected reliably, be- 
cause "high" will be input to the AND element even 
when one pixel is not functioning. 

Circuit 73b is connected to the addresses in mul- 
tiple line memory 72. It outputs to flip-flop group 73a 
the addresses of images which must be input, and it 
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outputs to the coordinates (XY) of the pixels output by 
the central flip-flop in group 73a. 

Let us assume that this scheme is being used and 
image data corresponding to Figure 46(A) is received. 
Marks 8a and 8b will be detected by search unit 73, 
and their coordinate values will be stored in location 
memory 81 . For the circles and squares, however, the 
output of decoder circuit 73 will go low; so their coor- 
dinate values will not be stored in memory 81. Figure 
46(A) shows how the locations of marks 8a and 8b, 
which constitute the specified pattern, are stored in 
location memory 81 as "1". The said circles and 
squares, then, can be used as a dummy pattern. 
Since triangles of a different size will not be detected 
either, they too can function as a dummy pattern. If 
such a dummy pattern is arranged in an appropriate 
fashion along the outline of the given shape or along 
some other outline, it will be more difficult for an un- 
informed person to determine the locations of the 
marks constituting the specified pattern. This wilt 
greatly reduce the probability of tampering. 

We shall now discuss device 80, the device which 
detects the specified pattern as shown in Figure 47. 
Detection device 80 consists of the aforesaid memory 
81, which stores the location of the marks, and device 
{here, CPU) 82, which, on the basis of the data stored 
in memory 81 , determines whether marks 8a and 8b, 
detected in the previous stage of processing, consti- 
tute the specified pattern. In response to a control 
signal from the program ROM 100, CPU 82 executes 
the processing shown in Figure 48. 

In this example the determination processing em- 
ploys a 1 5 x 15 window as shown in Figure 49. A de- 
termination as to whether the specified pattern has 
been found is made from the location of the marks in 
this window. Each mass in the window corresponds 
in size to a single mark. In this example, to cope with 
printing misregisters, detection errors and the like, a 
mass at a given distance from central mass C (a dis- 
tance which corresponds to the outline of a given 
shape) is given a coefficient (or a weight) of 1 ; a mass 
one unit removed from this location is given a coeffi- 
cient of 0.8, and a mass two units removed is given 
a coefficient of 0.5. 

CPU 82 reads out the data stored in location 
memory 81 and checks the masses in the window 
which correspond to the post ion coordinates of the 
marks. In this way every mass is checked for a mark. 
However, it is first determined whether there is a mark 
in centra! mass C (ST 1). If there is not, the specified 
pattern (whose center corresponds to the center of 
the window) cannot be in this window, so we shift the 
window in the direction of scanning (ST 8). 

If there is a mark in the central mass C in the win- 
dow, the marks in masses with coefficient 1 1 meaning 
that if there is a specif ied pattern, they are on its out- 
line, are counted; the marks in masses with coeffi- 
cient 0.8, meaning that they are one mass to the side, 



are counted; and the marks in masses with coeffi- 
cient 0.5, which are two masses to the side, are count- 
ed (ST 2 through ST 4). 

A total is obtained for all the marks counted as de- 

5 scribed above which have coefficients between 1 .0 
and 0.5, and it is determined whether this total falls in 
a given range (ST 5). If we assume, for example, that 
the specified pattern has ten marks along its outline, 
like that shown in Figure 44, we can obtain this range 

10 by adding and subtracting a fixed margin (the same 
value or different values may be used) to 10. 

If, in the determination in Step 5, the value falls 
outside the given range, it is concluded that the spe- 
cified pattern has not been found, and we jump to 

15 Step 8, move the window over, and proceed with the 
determination with respect to the next image. If the 
value does fall in the given range, the average value 
(the goodness of fit) of the said coefficients is calcu- 
lated, and this goodness of fit (or degree of resem- 

20 blance) is output (ST 6 and 7). 

As an example, let us assume that there is in the 
image a pattern identical to that shown in Figure 44. 
The position coordinates for every mark are stored in 
location memory 81 . If we develop these coordinates 

25 in a window, we get the window shown in Figure 50. 
Central mark 8a is in the central mass C of the said 
window, and there are ten marks along the periphery 
(the positions of the marks are indicated by Ms) . 
Since the coefficient of each mass where there is a 

30 mark is 1, the total in Step 2 will be 10, and the total 
in Steps 3 and 4 will be 0. The grand total will be 10, 
which is within the given range, so an average value 
for the coefficients will be obtained. Since in this case 
every value is 1, the average will also be 1 (indicating 

35 a goodness of fit of 1 t or 100%). If one of the marks 
is defaced and becomes illegible, the grand total will 
be 9. Since this is in the given range, an average val- 
ue will be calculated for the coefficients. The remain- 
ing nine marks all have a coefficient of 1 , so the aver- 

40 age value will be 1. 

In Figure 52, all the marks are in locations with a 
coefficient of 0.8. Since the total value for marks 
counted is 10, when we move to the processing to ob- 
tain an average value for the coefficients, we find that 

45 (0.8 x 10)/10 yields 0.8. 

Only marks with coefficients in the range be- 
tween 1 and 0.5 will be considered. Marks which are 
identical to marks 8a and 8b, which constitute the 
specified pattern, yet which are not in this range (i.e., 

so which have a coefficient of 0) will not be counted and 
will have no effect on the calculation of goodness of 
fit. Marks of the same shape as 8a and 8b found in 
locations with a coefficient of 0 constitute a dummy 
pattern. They make it more difficult for an uninformed 

55 person to determine which marks constitute the real 
pattern; and, in effect, they make it harder to tamper 
with the document. 

When the goodness of fit has been output we 
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move to Step 8 and shift the window by one mass in 
the direction of scanning. A determination is made as 
to whether shifting the window in the direction of 
scanning has caused it to exceed its own endpoint 
(ST 9). If it has not, we return to Step 1 and check the 
new window for the specified pattern (i.e., we deter- 
mine the goodness of fit on the basis of the marks 
constituting the pattern). 

If it is determined in Step 9 that the window has 
exceeded its own endpoint, we move back to the orig- 
inal position in the direction of scanning, and shift the 
window in the direction of feed. If the shift does not 
cause the window to exceed its endpoint in the direc- 
tion of feed, we return to Step 1 and execute the proc- 
essing to detect the specified pattern on the newly 
established window. In this way every region of the 
document which has been read will be checked. 

The goodness of fit output in Step 7 is transmitted 
to the copy machine or other device just as in the ex- 
ample we discussed. If the goodness of fit exceeds a 
given value, the copier processing will be suspended. 
In this example, the marks constituting the specified 
pattern on the document are detected, and a deter- 
mination is made as to whether this is the specified 
pattern based on the position coordinates of these 
marks. Thus all the data needed for a determination 
can be acquired on a single scan. With this scheme, 
the data can be processed at high speed. The pattern 
is not matched against a reference, so the determina- 
tion processing (i.e., the calculation of goodness of 
fit) can also be executed at high speed. 

In the examples discussed herein, the processing 
to detect the specified pattern is executed via soft- 
ware. However, the invention is not confined to this 
case, and nothing would prevent the processing to be 
executed by hardware. One example of the circuitry 
which might be used in this case is shown in Figure 
53. 

Flip-flop group 83 consists of 15 x 15 flip-flops 
which correspond to the aforesaid window. For each 
mark stored in location memory 81 , the flip-flop for its 
location is set to "1". If the data are set appropriately 
by memory 81 in flip-flop group 83, the window can 
be shifted via Steps 8 through 11 in Figure 48. 

The output of flip-flop group 83 (the output of 
each flip-flop) is sent to combination circuit 84. Circuit 
84 multiplies the output of each flip-flop by its respec- 
tive coefficient in Figure 49. By obtaining the average 
value of the result, it produces a value for goodness 
of fit. An example of how the circuits might be ar- 
ranged is shown in Figure 54. 

The output of each flip-flop in flip-flop group 83 
is sent to multiplier 84a. Multiplier 84a is connected to 
the output of each flip-flop. It multiplies that output by 
a given coefficient, which will have one of the values 
shown in Figure 49, either "0", "0.5", "0.8", or "1". The 
output of the central flip-flop will be multiplied by 0. 

Each output of multiplier 84a is input to counter 



84b, which counts the outputs of multiplier 84a which 
are not zero (0.5, 0.8, or 1). In this way, if the specified 
pattern is found in the window composed of flip-flop 
group 83, the total number of marks 8b on its periph- 
5 ery or within a given distance of it can be obtained 
(through the processing in Steps 2 through 4 in Figure 
48). 

The output of counter 84b is connected to com- 
parator 84c, a window comparator. The upper and 

w lower limit values, which serve as reference values, 
demarcate a range within which the aforesaid number 
of marks 8b must fall. If the output of counter 84b, 
which is the total number of marks 8b, falls within the 
reference range (i.e., the range between the upper 

15 and lower limit values), the output of the comparator 
will be 1 (H); if it does not, the output will be 0 (L). 

The output of comparator 84c is supplied to one 
of the input terminals of AND element 84d . The output 
of the central flip-flop in flip-flop group 83 (the flip- 

20 flop which corresponds to portion C in Figure 49) is 
connected to the other input terminal of AND element 
84d. If the central flip-flop is set to "1" (meaning that 
there is a mark 8a in the center) and the output of 
counter 84b (i.e., the total number of marks) is within 

25 the given range, the output of AND element 84d will 
be 1 (H); in all other cases, it will be 0 (L). AND ele- 
ment 84d processes the determinations in Steps 1 
and 5 of the flowchart shown in Figure 48. An output 
of 1 from element 84d indicates that the hit is valid (it 

30 is possible that the specified pattern has been 
found); an output of 0 indicates that it is invalid (the 
specified pattern is not there). 

The output of the said multiplier 84a is connected 
to adder 84e. Each output of multiplier 84a is added 

35 and a grand total is obtained. The output of adder 84e 
is connected to one of the input terminals of divider 
84f; the output of the said counter 84b is connected 
to the other input terminal. Divider 84f executes the di- 
vision of the output of adder 84e by the output of coun- 

40 ter 84b and outputs the result of its calculation. In this 
way the divider obtains an average value for the coef- 
ficients of the locations where marks 8b, the objects 
of the search, have been found, and thereby obtains 
the goodness of fit. This is the processing in Steps 6 

45 and 7 of the flowchart in Figure 48. 

As discussed above, along with the goodness of 
fit, the valid/invalid flag which is the output of AND 
element 84d is output. When the goodness of fit is 
greater than a given reference value and the "valid" 

so flag is output, the copy machine recognizes that the 
specified pattern has been found and executes its 
given program in response to this situation. 

In the example we have been discussing, multi- 
plier 84a is connected to the output of every flip-flop. 

55 However, one could also use a multiplier which did not 
multiply by coefficients which are zero. That is, the 
outputs of flip-flops corresponding to locations with 
coefficient 0 would not be connected to the multiplier. 

25 
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Also, in this example the output of multiplier 84a is 
connected to counter 84b; but the output of, for exam- 
ple, flip-flop group 83 could also be connected. In this 
case, only the outputs of flip-flops for locations with 
coefficients of 0.5, 0.8 or 1 would be connected to 
counter 84b. Then a flip-flop for a location with a mark 
would supply a "1" to counter 84b, while one for a lo- 
cation without a mark would supply a "0". An adder 
could be used to obtain the total number of flip-flops 
outputting a *T\ 

The flowchart in Figure 48 shows an example of 
the processing function being handled by the afore- 
said CPU 82. However, when the processing is exe- 
cuted via software, it will need to be compatible with 
that executed by the said hardware. The marks which 
have been detected can be used as a reference to 
execute the processing, as shown below. 

The position coordinates of the marks which have 
been detected are stored in location memory 81. 
From these coordinate data, we can obtain the dis- 
tance between any two marks. We choose one of the 
marks stored in location memory 81 and assume it is 
central mark 8a. We detect all the marks at a given 
distance from this mark (a given margin may also be 
provided) and obtain the number of marks which we 
have found. If the total is within a given range, we may 
determine that the specified pattern has been found. 
If, having used this device, we are able to predict that 
the number of marks which will be detected is small, 
we will be able to execute the relevant determination 
processing using a small number of circuits, which will 
contribute to high-speed processing. 

In the example we have been discussing, the spe- 
cified pattern consists of a central mark 8a and a 
number of identical marks 8b arrayed around this cen- 
tral mark at a given distance from it. However, the in- 
vention is not limited to this configuration only; a cen- 
tral mark 8a need not be used. In this case, the proc- 
essing in Step 1 of the flowchart in Figure 48 and AND 
element 84d in Figure 54 would be omitted. The de- 
vice would then be used in basically the same way as 
the other ideal examples discussed above. We would, 
however, be unable to execute via software the said 
determination processing using a mark as a refer- 
ence. For this reason it is preferable to have the spe- 
cified pattern consist of a central mark 8a and a set 
of marks 8b arranged around it. 

In the examples and modifications thereof dis- 
cussed herein, there is a single layer of marks 8b 
along the outline of the shape. However, the invention 
is not limited to this case only. A pattern could also be 
used (See Figure 55) in which a number of marks 8b 
of a given shape are arranged on the outlines of sev- 
eral shapes of varying sizes. 

In these examples, too, the determination of 
whether the specified pattern has been found is 
based on the number of marks 8b on the outline. The 
angular position (i.e., the location) of each mark could 



also be used as an additional criterion. 

The second example discussed above would de- 
tect pattern 8 in Figure 44, which has ten marks 8b, 
and pattern 8' in Figure 56, which also has ten marks 

5 8b\ as the same pattern. If the position of each mark 
is added as a criterion, however, they will be recog- 
nized as different The pattern in Figure 44 would be 
determined to the specified pattern, and that in Fig- 
ure 56 would not. Since the determination processing 

10 would make use of angular data, there would be little 
chance of false recognition lowering the processing 
speed, and the rate of recognition would improve. 

The device to process such a function can be the 
circuit disclosed in figure 57. The location of marks 

15 can be recognized by the same way as discussed in 
the second example, but some part of the recognition 
circuit for the specified pattern is different in this ex- 
ample. 

In this example, to guarantee that determinations 
20 can be processed at high speed, they are made (i.e., 
the goodness of fit is calculated) without using angu- 
lar position as a criterion, just as in the first example. 
If in the determination processing the goodness of fit 
is found to be high, the portion of the image contain- 
25 ing the pattern is extracted and matched against the 
reference pattern repeatedly after being rotated 
slightly. In this way the portion of the image can be 
extracted easily, and the number of times the pattern 
needs to be rotated and matched can be kept to a 
30 minimum. The processing can be executed at high 
speed. 

As a specific example of how such a device might 
be configured, we shall discuss a design realized 
through hardware. As can be seen in the drawing, lo- 

35 cation memory 81 , flip-flop group 83 and combination 
circuit 84, which serves as the device to calculate 
goodness of fit, are all taken without modification 
from the device in the prior example. The output of 
combination circuit 84 (i.e., the goodness of fit) is sent 

40 to circuit 85 (which could be an ordinary comparator), 
which checks the goodness of fit. This circuit deter- 
mines whether the goodness of fit exceeds a given 
reference value. If it does, the circuit sends a "request 
extraction" signal to control circuit 86, which will ex- 

45 tract the appropriate portion of the image. 

When control circuit 86 receives an extraction re- 
quest, it accesses from location memory 81 the pos- 
ition data concerning the given area where the mark 
has been found and having the same center point, 

so and it sends them to matching circuit 87. 

Matching circuit 87 compares the locations of the 
marks stored in memory unit 88, which stores the ref- 
erence pattern, with the detected pattern transmitted 
by control circuit 86, and it determines whether they 

55 coincide. It then rotates the detected pattern repeat- 
edly over a given angle until it has completed a 360° 
rotation. If at any angle the images match, the device 
determines that it has found the specified pattern. If 

26 
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they do not match at any angle, it concludes that it has 
not found the specified pattern. In either case, it out- 
puts the result. The goodness of fit synchronously 
calculated by combination circuit 84 could also be out- 
put at this time. The matching device consists of the 5 
said control circuit 86, which extracts the image, and 
matching circuit 87. 

In the scheme employed by this example, a mark 
is detected and the location of that mark is stored. 
These processes are the same as those used in the 10 
second example discussed earlier. The specified pat- 
tern, which consists of a number of marks arranged 
in a given relationship, is detected from the locations 
of the marks which are detected. First, it is determined 
whether this pattern has the same number of marks 15 
at a given radius as the specified pattern. This is its 
goodness of fit. The processing executed to calculate 
goodness of fit is the same as that used in the second 
example. If the resulting goodness of fit is high, the 
portion of the image containing the pattern is extract- 20 
ed and it is matched against a reference after being 
repeatedly rotated. If the specified pattern is detect- 
ed, this is communicated to the copy machine, and it 
executes its given processing to prevent the copying 
operation from proceeding. 25 

In all the examples and modifications thereof dis- 
cussed herein, there is a single pattern to be detect- 
ed. However, the invention is not limited to this case. 
The image processing device could also use a num- 
ber of specified patterns and be able to distinguish 30 
which one it had detected. This would allow data rep- 
resenting coded information to be added to the pat- 
tern. 

For example, several different patterns (differing 
in the number of marks 8b placed on the outline of the 35 
shape) like those shown in Figures 58 through 60 
could be established. By adding coded data to the 
patterns, we can cause each of them to convey a dif- 
ferent meaning. 

If a number of these specified patterns are used, 40 
they can be placed in various locations on a single 
document. Then if all the patterns or at least a given 
number of them are found on a document, the opera- 
tion of the copy machine or other device can be pro- 
hibited. When multiple checks are performed in this 45 
way, there will be a lower probability of recognition er- 
rors. If one of the patterns has become hard to detect 
because it has become abraded or soiled through 
long use, the document can still be detected via the 
remaining patterns. When a single pattern is used, so 
copying will be prevented if a pattern identical to one 
of the specified patterns happens to be found among 
the density data (in the characteristic count) from an 
ordinary document By requiring the matching of a 
number of specified patterns, we can reduce the 55 
chance that the copying of an ordinary document 
which should be allowed is prevented. 

The aforesaid coded data can be made to corre- 



spond to the type of bank note or stock certificate, or 
in the case of a bank note, to the denomination. Then 
when this system is operating and the copying proce- 
dure with respect to a prohibited document has been 
completed, the type of pattern detected and the fact 
that a stop command is issued can be stored, so that 
we can determine what type of documents are placed 
for copying in this device, or more specifically in the 
copier in which this device is installed. Various other 
schemes using coded data can also be offered. 

A specific example of this sort of device to detect 
a specified pattern is shown in Figure 61. Since the 
marks are detected in the same way as in the second 
example we discussed, the mark detection device of 
that example could also be used here without modi- 
fication. In this example we have modified one part 
of the device to detect the specified pattern. We shall 
discuss only the modified portion. As can be seen in 
Figure 61, value setting unit 84g, which sets upper 
and lower limit values, has been added to combina- 
tion circuit 84'. 

Value setting unit 84g sets as reference values in 
comparator 84c upper and lower limit values for each 
of patterns (A) through (C) in Figures 58 through 60. 
Pattern (A) in Figure 58 has three marks 8b; pattern 
(B) in Figure 59 has five; and pattern (C) in Figure 60 
has seven. Upper and lower limit values are set by 
adding or subtracting a given margin to the number of 
marks each pattern should have. In this example, we 
have used a margin of ±1 . Code is output along with 
the limit value data indicating for which pattern refer- 
ence values are being set. Another possible configur- 
ation for the circuits is that shown in Figure 54. Since 
the same numbering is used, we shall omit a discus- 
sion of the function of each component. 

In the image processing scheme used by this ex- 
ample, the marks are detected and their locations are 
stored. The processing used up to this point is identi- 
cal to that of the aforesaid second example. Next, the 
flip-flops in flip-flop group 83 which correspond to the 
locations of all the marks that are detected are set. 
Multipl ier 84a multiplies the output of each flip-flop by 
a given coefficient, and counter 84b counts the val- 
ues which are not zero. The outputs of multiplier 84a 
are added together by adder 84e to obtain the total of 
the coefficients, and this total is input into divider 84f 
just as in the second example. 

In this scheme, the upper and lower limit values 
which were established previously for the number of 
marks in each specified pattern are obtained succes- 
sively from value setting unit 84g. For each pattern, 
the goodness of fit is determined and a judgment of 
valid or invalid is rendered. A margin of, say, ±1 can 
be added to the actual number of marks. When, for 
example, specified pattern (B) is detected, the output 
of counter 84b will be "5". Value setting unit 84g pro- 
vided the upper and lower limits of "4" and "2" to com- 
parator 84c for pattern (A). The output of comparator 
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84c will be "0" (L), and AND element 84d will output 
an "invalid" flag. At the same time, value setting unit 
84g will output code indicating that (A) is the pattern 
in question. In this way we can know that the pattern 
we detected is not specified pattern (A). 5 

Next, value setting unit 84g sets *6 M and "4" in 
comparator 84c as the upper and lower limits for pat- 
tern (A). The output of comparator 84c will be "1" (H), 
and AND element 84d will output a "valid" flag. At the 
same time, value setting unit 84g will output code in- 10 
dicating that (B) is the pattern in question. Specified 
pattern (B), then is valid, and its goodness of fit is out- 
put by divider 84f. If the goodness of fit is high, it is 
determined that this is indeed specified pattern (B). 

Value setting unit 84g sets upper and lower limit 15 
values of "8" and "6" in comparator 84c for pattern (C). 
The output of comparator 84c will be "0" (L), and AND 
element 84d will output an "invalid" flag. At the same 
time, value setting unit 84g will output code indicating 
that (C) is the pattern in question. In this way we un- 20 
derstand that this is not specified pattern (C). This 
scheme, then, allows us to know which specified pat- 
tern has been detected. In response, the specified 
processing is executed. 

Other examples of patterns with coded informa- 25 
tion are patterns A, B and C in Figures 62, 63 and 64, 
respectively. All have the same number of marks 8b, 
but the placement of the marks and hence their rela- 
tionship with each other varies. These patterns can 
be specified and detected by the scheme shown in 30 
Figure 57, which uses rotational angle as a criterion. 

A specific design for a device to detect several 
specified patterns containing coded information is 
shown in Figure 65. (Although it is not shown in the 
drawing, the device to detect the marks used in the 35 
previous examples can be employed here without 
modification.) 

The configuration here is basically the same as 
that in Figure 57. The marks are detected by device 
2", and their locations are stored in location memory 40 
81. Based on the data representing these locations, 
every flip-flop in flip-flop group 83 is set to a specific 
value. The output of these flip-flops is sent to combin- 
ation circuit 84, and the goodness of fit is obtained. 
The goodness of fit which is calculated is sent to cir- 45 
cuit 85. If it exceeds a given reference value, a "re- 
quest extraction" signal is sent to control circuit 86. 

When control circuit 86 receives the request for 
extraction, it accesses the location data for the mark 
in the specified area stored in location memory 81 so 
which has the same center point, and it sends these 
data to matching circuit 87. 

Matching circuit 87 compares the pattern trans- 
mitted by control circuit 86 with a specified reference 
pattern. Reference patterns for each of patterns (A) 55 
through (C) are stored in dictionaries 88a through 
88c, respectively. Matching circuit 87 successively 
compares each of the three reference patterns in 88a 



through 88c with a single pattern which is detected, 
and determines whether it matches any of them. The 
extracted pattern is also rotated in fixed angular in- 
crements over 360°. If at any angle the extracted pat- 
tern matches a reference, the code for that reference 
pattern is output. 

If a match is not found at any angle, it is concluded 
that this is not the specified pattern, and this result is 
output. The goodness of fit calculated synchronously 
by combination circuit 84 can also be output at this 
time. 

The image processing scheme used in this exam- 
ple is basically identical to the one described above 
in which code is added to a specified pattern by vary- 
ing the number of marks. The aspect in which they dif- 
fer is how they determine which specified pattern has 
been found. In this scheme, it is determined whether 
the pattern has the same number of marks at a given 
radius as the specified pattern. This is its goodness 
of fit. If the resulting goodness of fit is high, the portion 
of the image containing the pattern is extracted and 
matched against a reference while being gradually ro- 
tated. Several reference patterns corresponding to 
the various specified patterns are used in this match- 
ing process. The extracted pattern is compared to 
each of the references, and if a match is found, the 
code for that reference pattern is output. In this way 
we can understand which specified pattern has been 
detected. The specified processing is executed in re- 
sponse to the detection of the proscribed document 
All of the examples discussed above (in which the 
specified pattern is composed of a number of identi- 
cal marks) are suitable for use in the copy machine 
pictured in Figures 26 and 27. 

In the examples discussed above, the device is 
installed in a copy machine; however, this invention is 
not limited to this application, but could be applied as 
well in a color scanner, a color printer, a fax machine, 
a device to transmit communications, or a variety of 
other devices. 

To give an example, a scanner employing this in- 
vention might have the configuration pictured in Fig- 
ure 66. The scanner comprises three main compo- 
nents: input unit 90, control unit 91 and output unit 92. 
Input unit 90 scans the light from a light source over 
a document The light reflected (or transmitted) by the 
document is detected by a photoelectric converter 
element such as a CCD, a photomultiplier, or a pho- 
todiode. This light is converted to an electrical signal 
which is transmitted to control unit 91 . Control unit 91 
amplifies the electrical signal from the input unit and 
executes various kinds of image processing, such as 
the correction of specified tones or contour enhance- 
ment, and transmits the corrected signal to output unit 
92. Output unit 92 transforms the data as needed, 
based on the signal it has received from control unit 
91, and outputs them to the specified output device. 
When the scanner and printer are separate devices 
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and the scanner does no more than read the docu- 
ment, the data will need to be sent from scanner to 
printer. It will thus be necessary that the image data 
which the scanner has read be stored temporarily in 
a memory device (an output device). Whatever proc- 5 
essing is required to write the data will be executed. 
If the output device is a printer or the like (which is in- 
stalled in the same device), the signal must be con- 
verted from an electrical to an optical signal, and the 
processing must be executed to convert the signal in 10 
order to write it on a paper medium (a photosensitive 
material). Since commonly available devices are 
used as the actual components, we shall omit a de- 
tailed discussion of them here. 

With this invention, an image processing device 15 
93 is provided. The signals representing the image 
data which are received from the said input unit 90 are 
input into image processing device 93 as well as into 
control unit 91. Image processing device 93 can em- 
ploy the processing devices in any of the examples 20 
discussed above, as well as the various devices to de- 
tect the specified pattern shown in the modified ver- 
sions thereof. Image processing device 93 executes 
the given processing (which is discussed above) re- 
quired to detect a specified pattern in the image data 25 
which it receives. If it does detect this pattern, it sends 
a "prohibit output" signal to control unit 91. In re- 
sponse to this signal, control unit 91 halts the signal 
output to output unit 92. This "prohibit output" signal 
could also be sent to input unit 90 or output unit 92. 30 

Figure 67 shows how this signal processing de- 
vice could be used in a printer. The image data (in the 
form of electrical signals) are sent to input unit 94 eith- 
er directly from the scanner or through a medium 
such as a storage device. Control unit 95 executes 35 
the processing necessary to convert the image (i.e., 
it converts the data to the form required by the output 
mechanism). Output unit 96 executes the processing 
required to convert the signal from electrical to opti- 
cal, and the image data are reproduced on a photo- 40 
sensitive material. 

If the scanner described above does not have an 
image processing device 93 with a device to halt its 
operation when a specified pattern has been detect- 
ed, the image data will be read even if the document 45 
is one which should not be read. 

The output signal from control unit 95 is sent to 
image processing device 97 (which is equivalent to 
the aforesaid image processing device 93), which 
executes the specified image processing and per- so 
forms the processing necessary to detect the speci- 
fied pattern. If the specified pattern is detected, an 
"output prohibited" signal is sent to control unit 95, 
and the output of data from unit 95 to output unit 96 
comes to a halt. An "operation prohibited" signal could 55 
also be sent directly to output unit 96 to cause that unit 
to halt its operation. 

As described above, the image processing 



scheme and device of this invention and the copy ma- 
chine in which they are employed do not depend on 
the overall size (i.e., the outline) in order to recognize 
a document which may not be copied (or printed). 
They focus on a specified pattern imprinted on a por- 
tion of the document and detect whether that pattern 
is present. Thus this scheme requires only a small 
memory capacity, allowing it to be realized at a low 
cost. The location of the specified pattern can be de- 
tected accurately on a single scan. The pattern can 
be extracted and the determination processing exe- 
cuted on subsequent scans or simultaneously with 
the same scan. 

If dummy marks which are identical or similar to 
the real ones are provided in the vicinity of the posi- 
tioning marks, it will be difficult for an uninformed per- 
son to distinguish which are the real positioning 
marks. This will minimize the probability of tampering. 
If the specified pattern is composed of marks which 
are identical or similar to the positioning marks, it will 
be even more difficult to tamper with the document. 
The same specified pattern can be used for a variety 
of documents. Since the device is provided with 
knowledge concerning this pattern, it will be able to 
detect any of the various documents which contain it. 
Then should any new document be issued which may 
not be copied, the same pattern can simply be im- 
printed on it, and it too will be detected. Since this 
scheme requires the storage of only a small amount 
of knowledge, a small memory capacity will suffice. 
Since only a small number of patterns need be proc- 
essed, the processing can be executed at high speed. 

If at least three positioning marks are arranged in 
specified positions, and the shape they form is an iso- 
sceles triangle or an asymmetrical polygon, the loca- 
tion of the pattern can be obtained simply, accurately 
and unmistakably from the locations of the marks. 
This will allow the determination of whether this is the 
specified pattern to be performed at high speed. 

If the device to detect the positioning marks con- 
sists of a device to convert to binary values the image 
data it receives and a device to search the binary data 
using a mask divided into n x m units, a simple circuit 
(hardware) can be used for detection, and the proc- 
essing can be executed at high speed. Since only a 
few positioning marks are needed, a compact and un- 
complicated device can be realized even if hardware 
is used. 

If the matching device calculates the degree of re- 
semblance, or more specifically, if it obtains the de- 
gree of resemblance by means of fuzzy inference, the 
pattern will be recognized correctly even when the 
actual pattern on the document does not mesh with 
the reference pattern because it is shifted slightly 
due to a misregister or other printing irregularity. By 
the same token, the use of fuzzy inference (fuzzy pat- 
tern matching) will prevent the accuracy of position- 
ing (and the recognition rate) from dropping when the 
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distance between the positioning marks and the pat- 
tern varies slightly. 

If the said matching device divides the extracted 
region into N cells and decodes each cell according 
to the density which it has been determined to have; 
and if it then determines whether the said decoded 
data match previously obtained reference data which 
represent the pattern to be detected, then the match- 
ing device can consist of a comparator which outputs 
a signal when a match has been found. The design of 
this device can be kept simple, and the determination 
processing can be executed at high speed. 

If positioning marks are not used, and the speci- 
fied pattern consists of a number of identical marks 
arranged in a given relationship, then in addition to 
the effects described above, the processing executed 
by the device to detect the marks can be virtually the 
same as the said processing used to detect the pos- 
itioning marks. Marks can be used which are identical 
to the said positioning marks. 

By installing this image processing device in a 
copy machine, we can absolutely prevent the output 
of copies of documents whose reproduction is illegal, 
such as bank notes and negotiable securities. (This 
entails refusing to copy the prohibited document 
while allowing other documents to be copied and out- 
put.) In addition, we can halt the reading or printing of 
proscribed documents. 



Claims 

1. An image processing device to detect a specified 
reference pattern, comprising: 

a pattern detection device to detect a po- 
tential pattern within an image data, said poten- 
tial pattern is a potential match for said specified 
reference pattern which is predetermined; 

a pattern extracting device to extract said 
detected potential pattern from said image data; 
and 

a pattern matching device to match said 
extracted potential pattern with said specified 
reference pattern; 

wherein said specified reference pattern 
comprises a number of cells having a character- 
istic. 

2. An image processing device according to claim 1 , 
wherein said characteristic comprises a color 
density made to differ from background of said 
specified reference pattern. 

3. An image processing device according claim 2, 
wherein said color density is a dots pattern. 

4. An image processing device according claim 2, 
wherein said color density represents coded in- 



formation. 

5. An image processing device according claim 1, 
wherein said pattern matching device performs a 

5 fuzzy inference using fuzzy knowledge assem- 

bled beforehand based on said specified refer- 
ence pattern, and calculates a goodness of fit of 
said potential pattern with respect to said speci- 
fied reference pattern. 

10 

6. An image processing device according claim 1, 
wherein said pattern detection device detects 
said potential pattern by a rough search which 
matches a gradated image obtained by reducing 

15 a resolution of said image data against a gradat- 

ed image of said specified reference pattern. 

7. An image processing device according to claim 2, 
wherein a number of different specified refer- 

20 ence patterns may be incorporated which have a 

common background having same coded infor- 
mation. 

8. An image processing device according to claim 1 , 
25 wherein specified reference pattern is made up 

of a number of cells arranged in a grid of dimen- 
sion n x m. 

9. An image processing device according to claim 1, 
30 wherein specified reference pattern is made up 

of a number of cells arranged in concentric cir- 
cles. 

1 0. An image processing device according to claim 2, 
35 wherein a virtual mark whose diameter is less 

than a width of said cell is established in a speci- 
fied location within said cell, and color density of 
said mark is varied. 

40 11. An image processing device according claim 2, 
wherein a sampling region used to obtain said 
color density of said cell is a region in a specified 
location in said cell which is within a circle whose 
diameter is less than a width of said cell. 

45 

12. An image processing device according claim 2, 
wherein said color density of one of said cells has 
a number of grades, so that said color density of 
said single cell can be expressed as multivalued 

so data. 

13. A copy machine to copy a transcript, comprising: 

a scanning device to scan said transcript 
and generate an image data of said transcript; 
55 a conversion device which is connected to 

said scanning device and which converts said im- 
age data into color signals for printing; 

a printing device, which receives said color 

30 
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signals from said conversion device and per- 
forms specified processing needed for printing; 

an image processing device to detect a 
specified reference pattern in said transcript, 
said image processing device is connected in 5 
parallel with said conversion device, comprising: 

a pattern detection device to detect 
a potential pattern within an image data, said po- 
tential pattern is a potential match for said speci- 
fied reference pattern which is predetermined; w 

a pattern extracting device to ex- 
tract said detected potential pattern from said im- 
age data; and 

a pattern matching device to match 
said extracted potential pattern with said speci- 15 
fied reference pattern; 

wherein said specified reference 
pattern comprising a number of cells having a 
characteristic; 

a control device to control said printing de- 20 
vice when it has determined that said specified 
reference pattern is present in said transcript in 
order to prohibit said specified processing need- 
ed for printing. 

25 

14. An image processing method to identify a speci- 
fied reference pattern, comprising steps of: 

detecting a plurality of location marks 
which are arranged in a first spatial relationship 
between each other in an image data; 30 

extracting a data region where said speci- 
fied reference pattern may exist, said data region 
is arranged in a second spatial relationship from 
said plurality of location marks; 

judging whether said region contains said 35 
specified reference pattern which identifies said 
image data. 

15. An image processing method to identify a speci- 
fied reference pattern, comprising steps of: 40 

detecting a plurality of identical specified 
reference marks which may be incorporated into 
said specified reference pattern in a image data; 

counting a total number of said specified 
reference marks which are arranged on a outline 45 
of a shape; and 

judging whether said specified reference 
pattern exists in said image data based on said 
total number of said reference marks. 

50 

16. An image processing method to identify a speci- 
fied reference pattern according to claim 14, 
wherein said image data consists of said plurality 
of location marks as well as dummy marks which 

are identical or similar to said location marks. 55 

17. An image processing method to identify a speci- 
fied reference pattern according to claim 15, 



wherein said image data consists of a said plur- 
ality of specif ied reference marks as well as dum- 
my marks which are identical or similar to said 
specified reference marks. 

18. An image processing method to identify a speci- 
fied reference pattern according to claim 14, 
wherein said specified reference pattern con- 
sists of marks that are identical or similar to said 
location marks, but which are arranged spatially 
in a relationship different from that of said loca- 
tion marks. 

19. An image processing method to identify a speci- 
fied reference pattern according to claim 15, 
wherein said specified reference pattern con- 
sists of marks that are identical or similar to said 
specified reference marks, but which are ar- 
ranged spatially in a relationship different from 
that of said specified reference marks. 

20. An image processing method to identify a speci- 
fied reference pattern according to claim 15, 
wherein said specified reference pattern consist- 
ing of a plurality of identical specified reference 
marks has an identical mark in the center of said 
shape. 

21. An image processing method to identify a speci- 
fied reference pattern according to claim 15, 
wherein said judging is done based on said total 
number of said reference marks and a relative lo- 
cation of each said reference marks on said out- 
line of shape. 

22. An image processing device to detect a specified 
reference pattern, comprising: 

a location mark detector to detect a loca- 
tion mark in a image data; 

a data region extracting device to extract 
a region of said specified reference pattern 
based on said location mark; and 

a pattern matching device to judge wheth- 
er an extracted pattern in said extracted region 
has said specified reference pattern. 

23. An image processing device to detect a specified 
reference pattern according to claim 22, wherein 
said location mark detector detects at least three 
location marks which are arranged in a predeter- 
mined spatial relationship. 

24. An image processing device to detect a specified 
reference pattern according to claim 23, wherein 
a shape formed by virtual lines linking at least 
three adjacent location marks arranged in a spe- 
cified relationship is an isosceles triangle or 
some other asymmetrical shape. 
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25. An image processing device to detect a specified 
reference pattern according to claim 22, wherein 
said location mark detector comprises: 

a binary converter to binary said image 
data having a plurality of mask partitioned into n 
x m; 

a location mark extracting device to com- 
pare said mask having a binary dark-light cells 
with said location mark, and extract said location 
mark; and 

a coordinate detector to detect a coordin- 
ate data of said location mark. 

26. An image processing device to detect a specified 
reference pattern according claim 22, wherein 
said pattern matching device performs a fuzzy in- 
ference using fuzzy knowledge assembled be- 
forehand based on said specified reference pat- 
tern, and calculates a goodness of fit of said ex- 
tracted pattern with respect to said specified ref- 
erence pattern. 

27. An image processing device to detect a specified 
reference pattern according to claim 22, wherein 
said pattern matching device decodes a color 
density of each ceil forming said data region, and 
judges whether said decoded color density is 
identical to said specified reference pattern. 

28. An image processing device to detect a specified 
reference pattern, comprising: 

a specified reference mark detector to de- 
tect identical specified reference marks which 
may be incorporated into said specified reference 
pattern in a image data; 

a counting device to count a total number 
of said specified reference marks which are ar- 
ranged on a outline of a shape; and 

a judging device to judge whether said 
specified reference pattern exists in said image 
data based on said total number of said reference 
marks. 

29. An image processing device to detect a specified 
reference pattern according to claim 28, wherein 
said specified reference marks are arranged on 
a outline of a plurality of shapes which are similar 
and different size, said specified reference 
marks on each shapes have different coeffi- 
cients which decreases with increasing said size, 
said judging considers both said total number of 
said specified reference marks and said coeffi- 
cients of said shape. 

30. An image processing device to detect a specified 
reference pattern according claim 28, wherein 
said judging device performs a fuzzy inference as 
a first step using fuzzy knowledge assembled be- 



forehand based on said specified reference pat- 
tern, and calculates a goodness of f it of said ex- 
tracted pattern with respect to said specified ref- 
erence pattern, then said judging device per- 
5 forms said judging as a second step only when 

said goodness of fit exceeds a specified value. 

31. A copy machine to copy a transcript, comprising: 

a scanning device to scan said transcript 
10 and generate an image data of said transcript 

a conversion device which is connected to 
said scanning device and which converts said im- 
age data into color signals for printing; 

a printing device, which receives said color 
15 signals from said conversion device and per- 

forms specified processing needed for printing; 

an image processing device to detect a 
specified reference pattern, comprising: 

a location mark detector to detect a 
20 location mark in a image data; 

a data region extracting device to 
extract a region of said specified reference pat- 
tern based on said location mark; 

a pattern matching device to judge 
25 whether a extracted pattern in said extracted re- 

gion has said specified reference pattern 

a control device to control said printing de- 
vice when it has determined that said specified 
reference pattern is present in said transcript in 
30 order to prohibit said specified processing need- 

ed for printing. 

32. A scanning machine to scan a transcript, com- 
prising: 

35 a scanning device to scan said transcript 

and generate an image data of said transcript; 

an image processing device to detect a 
specified reference pattern, comprising: 

a location mark detector to detect a 
40 location mark in a image data; 

a data region extracting device to 
extract a region of said specified reference pat- 
tern based on said location mark; 

a pattern matching device to judge 
45 whether a extracted pattern in said extracted re- 

gion has said specified reference pattern 

a control device to control said scanning 
machine when it has determined that said speci- 
fied reference pattern is present in said transcript 
so in order to prohibit said specified processing 

needed for scanning. 

33. A printing machine to print a transcript, compris- 
ing: 

55 a printing device, which receives color sig- 

nals from outside and performs specified proc- 
essing needed for printing; 

an image processing device to detect a 

32 
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specified reference pattern, comprising: 

a location mark detector to detect a 
location mark in a image data; 

a data region extracting device to 
extract a region of said specified reference pat- 
tern based on said location mark; 

a pattern matching device to judge 
whether a extracted pattern in said extracted re- 
gion has said specified reference pattern 

a control device to control said printing de- 
vice when it has determined that said specified 
reference pattern is present in said transcript in 
order to prohibit said specified processing need- 
ed for printing. 
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